-- Logs begin at Thu 2022-07-28 16:47:49 UTC, end at Thu 2022-07-28 18:38:33 UTC. -- Jul 28 17:51:39.556080 dv-cinder-846173-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 17:51:39.565498 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 17:51:39 2022] *** Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: nodename: dv-cinder-846173-4-iscsi Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: machine: x86_64 Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: clock source: unix Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: pcre jit disabled Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: detected number of CPU cores: 4 Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: current working directory: / Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: detected binary path: /usr/bin/uwsgi-core Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: your processes number limit is 47680 Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: your memory page size is 4096 bytes Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: detected max file descriptor number: 2048 Jul 28 17:51:39.567505 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: lock engine: pthread robust mutexes Jul 28 17:51:39.568273 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: thunder lock: enabled Jul 28 17:51:39.568273 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 17:51:39.568273 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 17:51:39.607250 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: Python main interpreter initialized at 0x55893f9419c0 Jul 28 17:51:39.607250 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: python threads support enabled Jul 28 17:51:39.607250 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: your server socket listen backlog is limited to 100 connections Jul 28 17:51:39.607250 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: your mercy for graceful operations on workers is 90 seconds Jul 28 17:51:39.607590 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 17:51:39.607667 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: *** Operational MODE: preforking *** Jul 28 17:51:39.607940 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: *** uWSGI is running in multiple interpreter mode *** Jul 28 17:51:39.607940 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: spawned uWSGI master process (pid: 80454) Jul 28 17:51:39.608143 dv-cinder-846173-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 17:51:39.609585 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: spawned uWSGI worker 1 (pid: 80463, cores: 1) Jul 28 17:51:39.609585 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: spawned uWSGI worker 2 (pid: 80464, cores: 1) Jul 28 17:51:39.609585 dv-cinder-846173-4-iscsi devstack@keystone.service[80454]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 17:51:45.162370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:51:45.162370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: from cryptography.utils import int_from_bytes Jul 28 17:51:45.163169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:51:45.163169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: from cryptography.utils import int_from_bytes Jul 28 17:51:45.163169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:51:45.163169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: from cryptography.utils import int_from_bytes Jul 28 17:51:45.163724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:51:45.163724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: from cryptography.utils import int_from_bytes Jul 28 17:51:45.212310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 17:51:45.212478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 17:51:45.212559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 17:51:45.212662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 17:51:45.212732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] command line args: [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 17:51:45.212829 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] command line args: [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 17:51:45.212958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 17:51:45.213130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ================================================================================ {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 17:51:45.213235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 17:51:45.213318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ================================================================================ {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 17:51:45.213395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] admin_token = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.213550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] admin_token = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.213630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] config_dir = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.213836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] config_dir = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.213934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.214008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] config_source = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.214077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.214147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] debug = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.214222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] config_source = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.214324 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] debug = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.214414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.214597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.214697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.214759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.214926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.215010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.215114 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.215188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.215362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.215447 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.215623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.215707 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.215767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_config_append = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.215825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_config_append = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.215939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.216031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.216107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_dir = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.216172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_dir = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.216301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_file = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.216381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_file = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.216505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.216591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.216754 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.216841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.217023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.217213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.217439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.217645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.217831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.217997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.218162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.218343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.218485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.218653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.218823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.218984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.219123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.219290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.219457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.219598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.219771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.220036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.220195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.220334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.220518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.220690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.221450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.221615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.230718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.230814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.230891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.86/identity {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.230982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.86/identity {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] publish_errors = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] publish_errors = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.231923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] standard_threads = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] standard_threads = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232185 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232541 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] use_journal = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] use_journal = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232705 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] use_json = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] use_json = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] use_stderr = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.232930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] use_stderr = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.233058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] use_syslog = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.233130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] use_syslog = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.233199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.233343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:51:45.233422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.233497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.233573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.233643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.233712 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.233788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.233864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.233976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234852 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.234989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235915 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.235990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.236911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.237942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238390 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238526 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.238954 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239276 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239400 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239522 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239816 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.239944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240502 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240849 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.240998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.241074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.241195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.241295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.241373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.241522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.241629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.241748 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.241862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.241936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.242067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.242142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.242265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.242337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.242406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.242525 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.242605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.242676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.242797 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.242879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.242958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.243067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.243144 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.243214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.243351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.243469 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.243581 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.243661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.243775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.243849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.243970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.244047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.244118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.244237 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.244312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.244425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.244574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.244648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.244762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.244836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.244948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245838 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.245988 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246102 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246850 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.246987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247542 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.247972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.248946 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249032 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249102 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249321 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.external = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.external = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249865 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.249924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.password = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250386 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.token = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250469 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.password = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.token = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.250964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251774 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251852 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.251924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252122 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.252847 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.253007 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.253353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.253538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.253618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.253780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.253862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.254041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.254119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.254328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.254406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.254575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.254656 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.254838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.254923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.255080 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.255164 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.255331 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.255410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.255568 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.255646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.255809 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.255893 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.256047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.256125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.256300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.256378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.256534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.256622 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.256788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.256873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.257045 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.257128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.257306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.257395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.257543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.257620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.257802 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.257880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.258041 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.258130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.258294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.258373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.258530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.258606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.258771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.258855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.259010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.259088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.259251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.259328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.259485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.259568 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.259725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.259803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.259965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.260041 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.260206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.260291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.260439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.260537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.260674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.260752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.260909 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.266357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.266494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.266572 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.266643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.266714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.266784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.266868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.266942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.267013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.267084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.267255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.267335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.267416 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.267490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.267560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268699 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.268919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269225 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269358 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.269927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270233 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270598 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.270964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.271943 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272817 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.272956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273262 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273610 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.273965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274533 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274648 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274866 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.274936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275324 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275531 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275885 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.275992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.276126 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.276229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.276366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.276478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.276605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.276730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.276853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.276984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.277132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.277250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.277373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.277484 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.277633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.277723 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.277867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.277942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.278168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.278245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.278409 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.278485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.278651 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.278788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.278891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.278970 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.279123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.279199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.279363 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.279446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.279601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.279678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.279846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.279960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.280089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.280169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.280324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.280401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.280579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.280663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.280817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.280895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.281088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.281167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.281341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.281430 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.281577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.281654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.281818 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.281897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.282055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.282140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] role.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.282298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.282376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] role.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.282528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.282604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] role.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.282778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] role.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.282862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.283014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.283092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.283309 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.283916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.284118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.284206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.284320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.284400 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.284571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.284649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.284822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.284909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.285099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.285181 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.285337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.285415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.285590 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.285674 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.285843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.285922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.286095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.286175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.286342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.286427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.286592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.286752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.286830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.286993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.287240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.287326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.287480 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.287558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.287732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.287808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.287985 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.288063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.288222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.288298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.288476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.288552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.288716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.288795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.288955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.289049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.289213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.289291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.289451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.289531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.289689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.289766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.289923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.289999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.290162 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.290243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.290416 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.290494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.290653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.290730 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.290893 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.290973 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.291129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.291206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.291367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.291443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.291601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.291681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.291840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.291917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.292076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.292152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.292327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.292405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.292568 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.292645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.292809 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.292885 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.293108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.293189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.293351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.293430 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.293598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.293675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.293844 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.293927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.294084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.294161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.294323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.294401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.294562 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.294640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.294801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.294879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.295034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.295109 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.295295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.295376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.295512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.295588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.295756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.295833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.connection = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.296051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.296141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.296301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.296378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.connection = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.296541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.296632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.296782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.296860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.297042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.297137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.297283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.297361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.297522 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.297599 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.297757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.297905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.297982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.298052 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.298221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.298300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.298462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.298547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.298698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.298776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.298970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.299048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.299212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.299298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.299467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.299545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.299710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.299788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.299956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.300042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.300191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.300268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.300362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 17:51:45.300486 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.300654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:51:45.300825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 17:51:45.329192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.329423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.329647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.329731 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.329939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.330694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.338569 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.339054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.368679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.368906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.369005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.369199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.422460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.426204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.474075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.474305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.475059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.475296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.539574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.539806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.540033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.540119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.540257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.540341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.540468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.540569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.540685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.540770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.540932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.541181 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.563609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.563776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.563863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.564011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.564110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.564235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.564328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.564454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.565063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.565229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.565345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.565424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.565498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.565632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.576458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.576704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.589670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.589907 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.590003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.590131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.590232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.590354 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.590439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.590569 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.590653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.590780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.590874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.590992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.591077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.591216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:51:45.607811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.608021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:51:45.608283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.608368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.608498 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:51:45.608582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:51:45.608810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.608893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.609054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:51:45.609141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.609326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.609410 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.609585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.609668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.609839 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.610096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.619325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.619606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.619873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.619958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.620152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.620235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.620421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.620706 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.627710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.628458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.632670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.632953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.633160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.633246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.633442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.633527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.633726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.633819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.634010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.634094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.634291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.634373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.634545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.634637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.634813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.634896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.635084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.635166 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.635351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.635435 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.635624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.635891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.659532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.659970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.661065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.661350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.661500 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.661784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.666003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.666286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.666568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.667041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.667498 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.667894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.673306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.673544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.675987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.676273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.678357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.678646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.678940 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.679232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.679510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.679803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.680082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.680351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.680500 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.680791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.681112 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.681402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.681684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.681976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.682257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.682537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.696343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.696625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.698569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.698852 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.700594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.703759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.704054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.704228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.707371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.707673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.708527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.711732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.712127 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.714922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.715227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.715359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.715513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.715807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.716096 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.716385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.718591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.718893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.719178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.719464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.719743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.720037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.732978 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.733263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.733531 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.733795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.737237 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.737555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.737831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.738093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.741440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.743002 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.743282 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.745061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.746125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.746408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.746617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.746702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.746897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.746982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.747265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.749766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.750050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.750354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.750657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.750940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.759094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.759374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.759664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.759951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.760234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.760512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.760790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.762730 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.763014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.763297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.763585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.763867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.764145 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.764445 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.776568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.779650 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.780178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.782659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.783176 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.785683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.786718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.787268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.787554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.787838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.789251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.790812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.791099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.791384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.794529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.796132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.797922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.799658 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.799974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.800117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.800200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.800476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.803105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.803396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.803684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.803969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.808635 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.808923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.809229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.812046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.812339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.812625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.816160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.816437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.816716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.817024 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.817313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.817603 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.817880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.818155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.818540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.818826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.819111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.819375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.819591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:51:45.819684 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.819875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.819967 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.820160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.820450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.820727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.821035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.821326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.821599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.821984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.822270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.822554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.822825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.823103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.823365 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:51:45.854803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55893f9419c0 pid: 80463 (default app) Jul 28 17:51:45.857939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55893f9419c0 pid: 80464 (default app) Jul 28 17:51:45.858208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a91707-3f48-45f5-b7b2-ad2bb396e18e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:45.858502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a91707-3f48-45f5-b7b2-ad2bb396e18e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:45.858780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a91707-3f48-45f5-b7b2-ad2bb396e18e None None] PATH_INFO: `/v3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:45.863624 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1/1] 10.222.0.86 () {50 vars in 791 bytes} [Thu Jul 28 17:51:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:51:54.573860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5950e1-1122-4ada-9849-5446fd232c09 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:54.574405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5950e1-1122-4ada-9849-5446fd232c09 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:54.574405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5950e1-1122-4ada-9849-5446fd232c09 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:54.578918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1/2] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:51:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:51:54.584400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890d4ad-05fe-4381-a953-f7477b7470c1 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:54.584822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890d4ad-05fe-4381-a953-f7477b7470c1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:54.584959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890d4ad-05fe-4381-a953-f7477b7470c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:54.655492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: INFO dbcounter [None req-a890d4ad-05fe-4381-a953-f7477b7470c1 None None] Registered counter for database keystone Jul 28 17:51:54.677576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG oslo_db.sqlalchemy.engines [None req-a890d4ad-05fe-4381-a953-f7477b7470c1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80463) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 17:51:54.683767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writer thread running {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 17:51:54.859788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG passlib.handlers.bcrypt [None req-a890d4ad-05fe-4381-a953-f7477b7470c1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80463) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 17:51:54.860206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG passlib.handlers.bcrypt [None req-a890d4ad-05fe-4381-a953-f7477b7470c1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80463) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 17:51:54.914906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a890d4ad-05fe-4381-a953-f7477b7470c1 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:51:54.916655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a890d4ad-05fe-4381-a953-f7477b7470c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:51:55.039268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 2/3] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:51:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 28 17:51:55.189874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5950e1-1122-4ada-9849-5446fd232c09 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:51:55.262096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: INFO dbcounter [None req-6f5950e1-1122-4ada-9849-5446fd232c09 None None] Registered counter for database keystone Jul 28 17:51:55.282296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG oslo_db.sqlalchemy.engines [None req-6f5950e1-1122-4ada-9849-5446fd232c09 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80464) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 17:51:55.289092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writer thread running {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 17:51:55.411286 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5950e1-1122-4ada-9849-5446fd232c09 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:51:55.415815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42505fc-1eb2-48a5-b876-17ce7fabab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:51:55.416452 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42505fc-1eb2-48a5-b876-17ce7fabab54 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:55.416737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42505fc-1eb2-48a5-b876-17ce7fabab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:55.417032 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42505fc-1eb2-48a5-b876-17ce7fabab54 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:55.425621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42505fc-1eb2-48a5-b876-17ce7fabab54 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:51:55.478881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d42505fc-1eb2-48a5-b876-17ce7fabab54 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 17:51:55.479222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42505fc-1eb2-48a5-b876-17ce7fabab54 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:51:55.493414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 2/4] 10.222.0.86 () {58 vars in 1324 bytes} [Thu Jul 28 17:51:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:51:55.499513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890d4ad-05fe-4381-a953-f7477b7470c1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:51:55.515281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890d4ad-05fe-4381-a953-f7477b7470c1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:51:55.520396 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f7682-7406-4407-a6c5-c6c2eabaee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:51:55.521542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f7682-7406-4407-a6c5-c6c2eabaee2e admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:55.521934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f7682-7406-4407-a6c5-c6c2eabaee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:55.522228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f7682-7406-4407-a6c5-c6c2eabaee2e admin admin] PATH_INFO: `/v3/projects/e2f95f7ebd0046e996236ac2938731ae` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:55.523731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f7682-7406-4407-a6c5-c6c2eabaee2e admin admin] RBAC: Authorizing `identity:get_project(project_id=e2f95f7ebd0046e996236ac2938731ae)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:51:55.582116 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fe1f7682-7406-4407-a6c5-c6c2eabaee2e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 17:51:55.582471 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f7682-7406-4407-a6c5-c6c2eabaee2e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:51:55.587252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 3/5] 10.222.0.86 () {56 vars in 1207 bytes} [Thu Jul 28 17:51:55 2022] GET /identity/v3/projects/e2f95f7ebd0046e996236ac2938731ae => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:51:57.552436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13e468-5527-4fcc-b0be-b57fe40aba20 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:57.553002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13e468-5527-4fcc-b0be-b57fe40aba20 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:57.553002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13e468-5527-4fcc-b0be-b57fe40aba20 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:57.557121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 3/6] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:51:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:51:57.562677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5de12c-da7c-4e69-8ba6-f2fc4e7c6017 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:57.563084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5de12c-da7c-4e69-8ba6-f2fc4e7c6017 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:57.563257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5de12c-da7c-4e69-8ba6-f2fc4e7c6017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:57.583127 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9c5de12c-da7c-4e69-8ba6-f2fc4e7c6017 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:51:57.584668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9c5de12c-da7c-4e69-8ba6-f2fc4e7c6017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:51:57.592696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 4/7] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:51:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 28 17:51:57.615031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13e468-5527-4fcc-b0be-b57fe40aba20 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:51:57.627345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13e468-5527-4fcc-b0be-b57fe40aba20 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:51:57.631741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f2b93-d79b-49fe-9afd-33cd4cfa1ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:51:57.632337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f2b93-d79b-49fe-9afd-33cd4cfa1ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:57.632619 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f2b93-d79b-49fe-9afd-33cd4cfa1ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:57.632891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f2b93-d79b-49fe-9afd-33cd4cfa1ed4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:57.640505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f2b93-d79b-49fe-9afd-33cd4cfa1ed4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:51:57.642293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f2b93-d79b-49fe-9afd-33cd4cfa1ed4 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:51:57.651513 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 4/8] 10.222.0.86 () {58 vars in 1324 bytes} [Thu Jul 28 17:51:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:51:57.656725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5de12c-da7c-4e69-8ba6-f2fc4e7c6017 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:51:57.665728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5de12c-da7c-4e69-8ba6-f2fc4e7c6017 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:51:57.670301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58806500-f126-4450-afdd-1bcd1b2c5b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:51:57.670881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58806500-f126-4450-afdd-1bcd1b2c5b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:57.671175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58806500-f126-4450-afdd-1bcd1b2c5b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:57.671452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58806500-f126-4450-afdd-1bcd1b2c5b60 admin admin] PATH_INFO: `/v3/users/1dff148688b84b08aaa5c8689269444f` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:57.672831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58806500-f126-4450-afdd-1bcd1b2c5b60 admin admin] RBAC: Authorizing `identity:get_user(user_id=1dff148688b84b08aaa5c8689269444f)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:51:57.674673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58806500-f126-4450-afdd-1bcd1b2c5b60 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:51:57.676206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 5/9] 10.222.0.86 () {56 vars in 1198 bytes} [Thu Jul 28 17:51:57 2022] GET /identity/v3/users/1dff148688b84b08aaa5c8689269444f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:51:59.723651 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211e1d6-0ba5-4e72-9f39-bc14ff77441a None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:59.724161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211e1d6-0ba5-4e72-9f39-bc14ff77441a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:59.724161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211e1d6-0ba5-4e72-9f39-bc14ff77441a None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:59.725512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 5/10] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:51:59.730664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5bbee-4b10-40b4-8146-5a2ed3fcfa36 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:59.731098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5bbee-4b10-40b4-8146-5a2ed3fcfa36 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:59.731229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5bbee-4b10-40b4-8146-5a2ed3fcfa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:59.750785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-fab5bbee-4b10-40b4-8146-5a2ed3fcfa36 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:51:59.752061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fab5bbee-4b10-40b4-8146-5a2ed3fcfa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:51:59.832450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 6/11] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:51:59.847061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c291338-49a8-42de-92e0-03dd50135cd6 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:59.847350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c291338-49a8-42de-92e0-03dd50135cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:59.847633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c291338-49a8-42de-92e0-03dd50135cd6 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:59.848997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 6/12] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:51:59.854139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5465f9-35ff-45b3-aead-7922c577b63f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:59.854431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5465f9-35ff-45b3-aead-7922c577b63f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:59.854707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5465f9-35ff-45b3-aead-7922c577b63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:59.854808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c291338-49a8-42de-92e0-03dd50135cd6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:51:59.873107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ea5465f9-35ff-45b3-aead-7922c577b63f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:51:59.874303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ea5465f9-35ff-45b3-aead-7922c577b63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:51:59.890576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:51:59.947065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 7/13] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:51:59.969342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5465f9-35ff-45b3-aead-7922c577b63f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:00.001490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c291338-49a8-42de-92e0-03dd50135cd6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:00.036118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1983b8d-2da1-46f0-abd0-43b05fa57c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:00.037016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1983b8d-2da1-46f0-abd0-43b05fa57c92 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:00.037312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1983b8d-2da1-46f0-abd0-43b05fa57c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:00.037597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1983b8d-2da1-46f0-abd0-43b05fa57c92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:00.049361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1983b8d-2da1-46f0-abd0-43b05fa57c92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a1983b8d-2da1-46f0-abd0-43b05fa57c92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:00.058206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:00.059106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:00.059976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 7/14] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:51:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:00.065382 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1983b8d-2da1-46f0-abd0-43b05fa57c92 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:00.069126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5465f9-35ff-45b3-aead-7922c577b63f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:00.100507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:00.105409 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e995bc5-6839-48cf-833f-48dd4e1690dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:00.106049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e995bc5-6839-48cf-833f-48dd4e1690dc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:00.106334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e995bc5-6839-48cf-833f-48dd4e1690dc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:00.106608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e995bc5-6839-48cf-833f-48dd4e1690dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:00.128250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:00.175965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1983b8d-2da1-46f0-abd0-43b05fa57c92 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:00.180798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e995bc5-6839-48cf-833f-48dd4e1690dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:00.182312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e995bc5-6839-48cf-833f-48dd4e1690dc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:00.210288 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195e90c-457e-448d-88ef-b3c89f1680c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:00.210918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195e90c-457e-448d-88ef-b3c89f1680c4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:00.211207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195e90c-457e-448d-88ef-b3c89f1680c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:00.211491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195e90c-457e-448d-88ef-b3c89f1680c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:00.212230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195e90c-457e-448d-88ef-b3c89f1680c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:00.212540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195e90c-457e-448d-88ef-b3c89f1680c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:00.213960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195e90c-457e-448d-88ef-b3c89f1680c4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:00.226065 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 8/15] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:00.231565 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195e90c-457e-448d-88ef-b3c89f1680c4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:00.278951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 8/16] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:51:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:52:00.284878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e995bc5-6839-48cf-833f-48dd4e1690dc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:00.325563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:00.331929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195e90c-457e-448d-88ef-b3c89f1680c4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:00.335197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:00.366911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504183d-af02-4ab5-a3c3-51372cb6f0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:00.367561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504183d-af02-4ab5-a3c3-51372cb6f0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:00.367844 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504183d-af02-4ab5-a3c3-51372cb6f0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:00.368120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504183d-af02-4ab5-a3c3-51372cb6f0c9 None admin] PATH_INFO: `/v3/users/1dff148688b84b08aaa5c8689269444f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:00.369750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504183d-af02-4ab5-a3c3-51372cb6f0c9 None admin] RBAC: Authorizing `identity:get_user(user_id=1dff148688b84b08aaa5c8689269444f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:00.371190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504183d-af02-4ab5-a3c3-51372cb6f0c9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:00.372998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 9/17] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:00 2022] GET /identity/v3/users/1dff148688b84b08aaa5c8689269444f => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:00.378658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504183d-af02-4ab5-a3c3-51372cb6f0c9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:00.382237 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e995bc5-6839-48cf-833f-48dd4e1690dc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:00.416451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addd200-5a9e-4d9c-a6ca-f1a782642763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:00.417141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addd200-5a9e-4d9c-a6ca-f1a782642763 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:00.417431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addd200-5a9e-4d9c-a6ca-f1a782642763 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:00.417712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addd200-5a9e-4d9c-a6ca-f1a782642763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:00.486189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504183d-af02-4ab5-a3c3-51372cb6f0c9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:00.523725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d701a0-ceb3-46c3-a7fc-305d109fad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:00.524384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d701a0-ceb3-46c3-a7fc-305d109fad4b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:00.524661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d701a0-ceb3-46c3-a7fc-305d109fad4b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:00.524932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d701a0-ceb3-46c3-a7fc-305d109fad4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:00.526232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d701a0-ceb3-46c3-a7fc-305d109fad4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:00.527695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d701a0-ceb3-46c3-a7fc-305d109fad4b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:00.529506 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 10/18] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:00.534830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addd200-5a9e-4d9c-a6ca-f1a782642763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:00.535124 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d701a0-ceb3-46c3-a7fc-305d109fad4b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:00.536287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addd200-5a9e-4d9c-a6ca-f1a782642763 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-2addd200-5a9e-4d9c-a6ca-f1a782642763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:52:00.555440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:00.556383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:00.557439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:52:00.557439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:52:00.557439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:52:00.557439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:00.557439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 9/19] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:00.563083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addd200-5a9e-4d9c-a6ca-f1a782642763 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:00.571473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:00.639336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d701a0-ceb3-46c3-a7fc-305d109fad4b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:00.662726 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addd200-5a9e-4d9c-a6ca-f1a782642763 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:00.664937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:00.674311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786592e8-94a1-4fb4-8f1e-c2aaab26495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:00.674935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786592e8-94a1-4fb4-8f1e-c2aaab26495d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:00.675219 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786592e8-94a1-4fb4-8f1e-c2aaab26495d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:00.678882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786592e8-94a1-4fb4-8f1e-c2aaab26495d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:00.678882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786592e8-94a1-4fb4-8f1e-c2aaab26495d None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.domain.id=default, user.id=1dff148688b84b08aaa5c8689269444f) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:00.678882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786592e8-94a1-4fb4-8f1e-c2aaab26495d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:00.678882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786592e8-94a1-4fb4-8f1e-c2aaab26495d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:00.686204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 11/20] 10.222.0.86 () {56 vars in 1322 bytes} [Thu Jul 28 17:52:00 2022] GET /identity/v3/role_assignments?user.id=1dff148688b84b08aaa5c8689269444f&scope.domain.id=default&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:00.697490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd020ecc-1ebe-4723-8e24-861c74d34ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:00.698140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd020ecc-1ebe-4723-8e24-861c74d34ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:00.698421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd020ecc-1ebe-4723-8e24-861c74d34ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:00.698694 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd020ecc-1ebe-4723-8e24-861c74d34ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:00.699415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd020ecc-1ebe-4723-8e24-861c74d34ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:00.699720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd020ecc-1ebe-4723-8e24-861c74d34ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:00.701198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd020ecc-1ebe-4723-8e24-861c74d34ba4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:00.721353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 10/21] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:02.942525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229589e2-6ec7-470b-9d94-953146d86a2a None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:02.943568 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229589e2-6ec7-470b-9d94-953146d86a2a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:02.944063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229589e2-6ec7-470b-9d94-953146d86a2a None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:02.945658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 12/22] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:02.952293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5515c-c1eb-48e2-8d89-0b2c101bc3fb None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:02.952734 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5515c-c1eb-48e2-8d89-0b2c101bc3fb None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:02.952915 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5515c-c1eb-48e2-8d89-0b2c101bc3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:02.982347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-fac5515c-c1eb-48e2-8d89-0b2c101bc3fb None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:02.983559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fac5515c-c1eb-48e2-8d89-0b2c101bc3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:03.080642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 11/23] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:03.118514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229589e2-6ec7-470b-9d94-953146d86a2a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:03.259726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:03.267924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229589e2-6ec7-470b-9d94-953146d86a2a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:03.320724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198ae4b-f0d9-4188-bace-b98e42189f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:03.321413 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198ae4b-f0d9-4188-bace-b98e42189f5a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:03.321751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198ae4b-f0d9-4188-bace-b98e42189f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:03.322035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198ae4b-f0d9-4188-bace-b98e42189f5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:03.340189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198ae4b-f0d9-4188-bace-b98e42189f5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3198ae4b-f0d9-4188-bace-b98e42189f5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:03.345322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:03.346152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:03.346915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 13/24] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:52:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:03.352836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5515c-c1eb-48e2-8d89-0b2c101bc3fb None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:03.355729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba852b5-50c0-4dfd-a3fd-14a090373507 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:03.356021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba852b5-50c0-4dfd-a3fd-14a090373507 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:03.356307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba852b5-50c0-4dfd-a3fd-14a090373507 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:03.357811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 14/25] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:52:03.365892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fa049-ccaf-43c2-8452-c093ef336ea7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:03.366386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fa049-ccaf-43c2-8452-c093ef336ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:03.366804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fa049-ccaf-43c2-8452-c093ef336ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:03.395746 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG passlib.handlers.bcrypt [None req-386fa049-ccaf-43c2-8452-c093ef336ea7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80464) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 17:52:03.396391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG passlib.handlers.bcrypt [None req-386fa049-ccaf-43c2-8452-c093ef336ea7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80464) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 17:52:03.478075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-386fa049-ccaf-43c2-8452-c093ef336ea7 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:03.479693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-386fa049-ccaf-43c2-8452-c093ef336ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:03.570715 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:03.587129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5515c-c1eb-48e2-8d89-0b2c101bc3fb None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:03.659415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 15/26] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:03.661108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61b2a2-ef1c-4e43-bfda-ad6f8d2f382a None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:03.661439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61b2a2-ef1c-4e43-bfda-ad6f8d2f382a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:03.661688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61b2a2-ef1c-4e43-bfda-ad6f8d2f382a None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:03.663330 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 16/27] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:03.668450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00718a2-3cb8-4790-90fc-57729895e69c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:03.668748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00718a2-3cb8-4790-90fc-57729895e69c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:03.669060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00718a2-3cb8-4790-90fc-57729895e69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:03.670223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fecdec-1ef2-4193-b873-aa69a339d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:03.670872 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fecdec-1ef2-4193-b873-aa69a339d53a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:03.671150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fecdec-1ef2-4193-b873-aa69a339d53a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:03.671418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fecdec-1ef2-4193-b873-aa69a339d53a None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:03.672153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fecdec-1ef2-4193-b873-aa69a339d53a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:03.672459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fecdec-1ef2-4193-b873-aa69a339d53a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:03.674435 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fecdec-1ef2-4193-b873-aa69a339d53a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:03.696339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 12/28] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:03.696983 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fecdec-1ef2-4193-b873-aa69a339d53a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:03.703516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c00718a2-3cb8-4790-90fc-57729895e69c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:03.704753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c00718a2-3cb8-4790-90fc-57729895e69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:03.800593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 17/29] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:03.801992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00718a2-3cb8-4790-90fc-57729895e69c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:03.822313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fecdec-1ef2-4193-b873-aa69a339d53a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:03.823672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:03.869445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fd17d-d1f4-4997-9d05-1c6a9f826d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:03.870404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fd17d-d1f4-4997-9d05-1c6a9f826d5e None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:03.870695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fd17d-d1f4-4997-9d05-1c6a9f826d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:03.870982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fd17d-d1f4-4997-9d05-1c6a9f826d5e None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:03.871975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fd17d-d1f4-4997-9d05-1c6a9f826d5e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:03.874144 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fd17d-d1f4-4997-9d05-1c6a9f826d5e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:03.915710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00718a2-3cb8-4790-90fc-57729895e69c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:04.060663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d109adf-ee98-49d6-bec7-dd58f9d73c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:04.062026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d109adf-ee98-49d6-bec7-dd58f9d73c78 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:04.062590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d109adf-ee98-49d6-bec7-dd58f9d73c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:04.063042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d109adf-ee98-49d6-bec7-dd58f9d73c78 None admin] PATH_INFO: `/v3/users/1dff148688b84b08aaa5c8689269444f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:04.065361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d109adf-ee98-49d6-bec7-dd58f9d73c78 None admin] RBAC: Authorizing `identity:get_user(user_id=1dff148688b84b08aaa5c8689269444f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:04.067069 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d109adf-ee98-49d6-bec7-dd58f9d73c78 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:04.068943 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 18/30] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:03 2022] GET /identity/v3/users/1dff148688b84b08aaa5c8689269444f => generated 247 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:04.070419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2705b7-877b-488f-a16f-3a28b451db66 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:04.070887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2705b7-877b-488f-a16f-3a28b451db66 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:04.071301 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2705b7-877b-488f-a16f-3a28b451db66 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:04.072733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 19/31] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:04.073917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2705b7-877b-488f-a16f-3a28b451db66 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:04.107167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 13/32] 10.222.0.86 () {60 vars in 1131 bytes} [Thu Jul 28 17:52:03 2022] POST /identity/v3/roles => generated 219 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:04.108760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fd17d-d1f4-4997-9d05-1c6a9f826d5e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:04.145590 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:04.234741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:04.269102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2705b7-877b-488f-a16f-3a28b451db66 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:04.322041 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fd17d-d1f4-4997-9d05-1c6a9f826d5e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:04.327631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05188584-78d4-42c4-809e-e177e0f535dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:04.327768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05188584-78d4-42c4-809e-e177e0f535dd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:04.327992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05188584-78d4-42c4-809e-e177e0f535dd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:04.328268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05188584-78d4-42c4-809e-e177e0f535dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:04.350310 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05188584-78d4-42c4-809e-e177e0f535dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:04.352309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05188584-78d4-42c4-809e-e177e0f535dd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:04.370577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9efbe9-193b-4961-ad1f-37061c5d07c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:04.371284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9efbe9-193b-4961-ad1f-37061c5d07c7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:04.371573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9efbe9-193b-4961-ad1f-37061c5d07c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:04.371859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9efbe9-193b-4961-ad1f-37061c5d07c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-05188584-78d4-42c4-809e-e177e0f535dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:52:04.372350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:04.373153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:04.373998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:52:04.373998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:52:04.373998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:52:04.373998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:04.373998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 20/33] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:04.375178 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9efbe9-193b-4961-ad1f-37061c5d07c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:04.375710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e7ddb-deb7-4279-bdd3-2d67db6e1d4d None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:04.376011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e7ddb-deb7-4279-bdd3-2d67db6e1d4d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:04.376295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e7ddb-deb7-4279-bdd3-2d67db6e1d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:04.378249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9efbe9-193b-4961-ad1f-37061c5d07c7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:04.381370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 14/34] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:04.382700 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffc12d-9b0e-4ab4-a4fe-44a8bf2b7b0c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:04.382991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffc12d-9b0e-4ab4-a4fe-44a8bf2b7b0c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:04.383273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffc12d-9b0e-4ab4-a4fe-44a8bf2b7b0c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:04.388744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 15/35] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:04.389622 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffc12d-9b0e-4ab4-a4fe-44a8bf2b7b0c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:04.399385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9e1e7ddb-deb7-4279-bdd3-2d67db6e1d4d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:04.402368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9e1e7ddb-deb7-4279-bdd3-2d67db6e1d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:04.478307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 21/36] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:04.479472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e7ddb-deb7-4279-bdd3-2d67db6e1d4d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:04.495524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:04.514082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffc12d-9b0e-4ab4-a4fe-44a8bf2b7b0c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:04.549193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2437d-b005-4ed9-a724-cdce06261b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:04.549831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2437d-b005-4ed9-a724-cdce06261b89 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:04.550120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2437d-b005-4ed9-a724-cdce06261b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:04.550401 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2437d-b005-4ed9-a724-cdce06261b89 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:04.551111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2437d-b005-4ed9-a724-cdce06261b89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:04.551427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2437d-b005-4ed9-a724-cdce06261b89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:04.552823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2437d-b005-4ed9-a724-cdce06261b89 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:04.565140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 16/37] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:04.566347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dd936-d619-46a1-a17b-d7d3df076435 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:04.566639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dd936-d619-46a1-a17b-d7d3df076435 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:04.566925 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dd936-d619-46a1-a17b-d7d3df076435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:04.570325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:04.587456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1d9dd936-d619-46a1-a17b-d7d3df076435 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:04.589019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1d9dd936-d619-46a1-a17b-d7d3df076435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:04.610073 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e7ddb-deb7-4279-bdd3-2d67db6e1d4d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:04.647845 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66d06e-190f-43c6-9c7c-8259c2bb3a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:04.648529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66d06e-190f-43c6-9c7c-8259c2bb3a80 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:04.648834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66d06e-190f-43c6-9c7c-8259c2bb3a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:04.649139 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66d06e-190f-43c6-9c7c-8259c2bb3a80 None admin] PATH_INFO: `/v3/domains/default/users/1dff148688b84b08aaa5c8689269444f/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:04.651207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da66d06e-190f-43c6-9c7c-8259c2bb3a80 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1dff148688b84b08aaa5c8689269444f, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:04.652672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da66d06e-190f-43c6-9c7c-8259c2bb3a80 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:04.664672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 17/38] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:04.665552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dd936-d619-46a1-a17b-d7d3df076435 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:04.740231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:04.765367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dd936-d619-46a1-a17b-d7d3df076435 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:04.798095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e215d76-d9f5-426a-828c-a5c2fcdc0287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:04.799243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e215d76-d9f5-426a-828c-a5c2fcdc0287 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:04.799532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e215d76-d9f5-426a-828c-a5c2fcdc0287 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:04.799814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e215d76-d9f5-426a-828c-a5c2fcdc0287 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:04.800533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e215d76-d9f5-426a-828c-a5c2fcdc0287 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:04.802195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e215d76-d9f5-426a-828c-a5c2fcdc0287 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:04.851830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 22/39] 10.222.0.86 () {58 vars in 1361 bytes} [Thu Jul 28 17:52:04 2022] PUT /identity/v3/domains/default/users/1dff148688b84b08aaa5c8689269444f/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:04.853089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66d06e-190f-43c6-9c7c-8259c2bb3a80 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:04.889010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 18/40] 10.222.0.86 () {60 vars in 1131 bytes} [Thu Jul 28 17:52:04 2022] POST /identity/v3/roles => generated 225 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:04.889010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e215d76-d9f5-426a-828c-a5c2fcdc0287 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:04.991833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66d06e-190f-43c6-9c7c-8259c2bb3a80 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:05.016421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e215d76-d9f5-426a-828c-a5c2fcdc0287 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:05.023970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:05.038887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee49e70-ca78-46f9-9700-b2cb32fd35b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:05.039510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee49e70-ca78-46f9-9700-b2cb32fd35b7 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:05.039874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee49e70-ca78-46f9-9700-b2cb32fd35b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:05.040164 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee49e70-ca78-46f9-9700-b2cb32fd35b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:05.040913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee49e70-ca78-46f9-9700-b2cb32fd35b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:05.042677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee49e70-ca78-46f9-9700-b2cb32fd35b7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:05.059604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a89f39-24db-4390-9932-302e76de4369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:05.060544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a89f39-24db-4390-9932-302e76de4369 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:05.060834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a89f39-24db-4390-9932-302e76de4369 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:05.061257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a89f39-24db-4390-9932-302e76de4369 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:05.062492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a89f39-24db-4390-9932-302e76de4369 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:05.064049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a89f39-24db-4390-9932-302e76de4369 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:05.067927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:05.170102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 23/41] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:52:04 2022] POST /identity/v3/projects => generated 301 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:05.171391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983fa9d-2099-422c-a7a5-0701f569cf2a None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:05.171694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983fa9d-2099-422c-a7a5-0701f569cf2a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:05.171982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983fa9d-2099-422c-a7a5-0701f569cf2a None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:05.174014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 24/42] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:05.182926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c8586-64f7-4e85-b2b2-d1cc60de93cc None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:05.183232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c8586-64f7-4e85-b2b2-d1cc60de93cc None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:05.183560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c8586-64f7-4e85-b2b2-d1cc60de93cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:05.203754 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7f9c8586-64f7-4e85-b2b2-d1cc60de93cc None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:05.205418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7f9c8586-64f7-4e85-b2b2-d1cc60de93cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:05.251208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 19/43] 10.222.0.86 () {60 vars in 1131 bytes} [Thu Jul 28 17:52:04 2022] POST /identity/v3/roles => generated 223 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:05.293974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 25/44] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:05.318395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a89f39-24db-4390-9932-302e76de4369 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:05.446634 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a89f39-24db-4390-9932-302e76de4369 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:05.484708 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e008294-1ed4-4ae3-8d69-cd6c5263c124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:05.485404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e008294-1ed4-4ae3-8d69-cd6c5263c124 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:05.485695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e008294-1ed4-4ae3-8d69-cd6c5263c124 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:05.486007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e008294-1ed4-4ae3-8d69-cd6c5263c124 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:05.487238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e008294-1ed4-4ae3-8d69-cd6c5263c124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:05.488695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e008294-1ed4-4ae3-8d69-cd6c5263c124 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:05.490337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 20/45] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:05.496057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c8586-64f7-4e85-b2b2-d1cc60de93cc None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:05.596778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c8586-64f7-4e85-b2b2-d1cc60de93cc None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:05.606123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:05.630575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc3100-57a5-43ac-97a5-4ff7fe8c59f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:05.631212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc3100-57a5-43ac-97a5-4ff7fe8c59f1 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:05.631503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc3100-57a5-43ac-97a5-4ff7fe8c59f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:05.631789 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc3100-57a5-43ac-97a5-4ff7fe8c59f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:05.632504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc3100-57a5-43ac-97a5-4ff7fe8c59f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:05.634007 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc3100-57a5-43ac-97a5-4ff7fe8c59f1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:05.760609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 26/46] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:52:05 2022] POST /identity/v3/projects => generated 312 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:52:06.922456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c09d12-60f7-4795-8f61-8f468e2817fd None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:06.923014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c09d12-60f7-4795-8f61-8f468e2817fd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:06.923014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c09d12-60f7-4795-8f61-8f468e2817fd None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:06.924588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 21/47] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:06.929973 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cf8eb-0bd6-4596-a53b-a7ef70d2da86 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:06.930388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cf8eb-0bd6-4596-a53b-a7ef70d2da86 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:06.930539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cf8eb-0bd6-4596-a53b-a7ef70d2da86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:06.949345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-513cf8eb-0bd6-4596-a53b-a7ef70d2da86 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:06.950560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-513cf8eb-0bd6-4596-a53b-a7ef70d2da86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:07.016573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 27/48] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:07.039977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c09d12-60f7-4795-8f61-8f468e2817fd None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:07.102966 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:07.152416 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c09d12-60f7-4795-8f61-8f468e2817fd None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:07.190286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f3e0e-f9c3-464b-92a8-f592c42dc34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:07.190858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f3e0e-f9c3-464b-92a8-f592c42dc34c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:07.191148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f3e0e-f9c3-464b-92a8-f592c42dc34c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:07.191432 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f3e0e-f9c3-464b-92a8-f592c42dc34c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:07.202996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f3e0e-f9c3-464b-92a8-f592c42dc34c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ae2f3e0e-f9c3-464b-92a8-f592c42dc34c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:07.209735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:07.210646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:07.211412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 22/49] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:52:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:07.216794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cf8eb-0bd6-4596-a53b-a7ef70d2da86 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:07.303829 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:07.313276 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cf8eb-0bd6-4596-a53b-a7ef70d2da86 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:07.347539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c93e9-6d18-4b71-a3de-c167623bb458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:07.348172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c93e9-6d18-4b71-a3de-c167623bb458 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:07.348463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c93e9-6d18-4b71-a3de-c167623bb458 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:07.348743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c93e9-6d18-4b71-a3de-c167623bb458 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:07.349504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c93e9-6d18-4b71-a3de-c167623bb458 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:07.349818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c93e9-6d18-4b71-a3de-c167623bb458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:07.351195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c93e9-6d18-4b71-a3de-c167623bb458 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:07.360068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 28/50] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:07.364817 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f3e0e-f9c3-464b-92a8-f592c42dc34c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:07.465140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f3e0e-f9c3-464b-92a8-f592c42dc34c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:07.486343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:07.498237 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7139b9-c8f7-4318-a59e-3bc724f8135b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:07.498881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7139b9-c8f7-4318-a59e-3bc724f8135b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:07.499166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7139b9-c8f7-4318-a59e-3bc724f8135b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:07.499440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7139b9-c8f7-4318-a59e-3bc724f8135b None admin] PATH_INFO: `/v3/users/1dff148688b84b08aaa5c8689269444f` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:07.500618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7139b9-c8f7-4318-a59e-3bc724f8135b None admin] RBAC: Authorizing `identity:get_user(user_id=1dff148688b84b08aaa5c8689269444f)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:07.502884 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7139b9-c8f7-4318-a59e-3bc724f8135b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:07.504630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 23/51] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:07 2022] GET /identity/v3/users/1dff148688b84b08aaa5c8689269444f => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:07.510110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c93e9-6d18-4b71-a3de-c167623bb458 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:07.620410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c93e9-6d18-4b71-a3de-c167623bb458 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:07.654392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bbfe6-1dac-4650-b732-71a9bee40082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:07.655041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bbfe6-1dac-4650-b732-71a9bee40082 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:07.655326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bbfe6-1dac-4650-b732-71a9bee40082 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:07.655603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bbfe6-1dac-4650-b732-71a9bee40082 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:07.656754 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bbfe6-1dac-4650-b732-71a9bee40082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:07.658338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bbfe6-1dac-4650-b732-71a9bee40082 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:07.659941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 29/52] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:07.665626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7139b9-c8f7-4318-a59e-3bc724f8135b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:07.764843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7139b9-c8f7-4318-a59e-3bc724f8135b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:07.796949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37451b6d-a2ed-44ec-8b04-3ca0a3a6b1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:07.797586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37451b6d-a2ed-44ec-8b04-3ca0a3a6b1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:07.797869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37451b6d-a2ed-44ec-8b04-3ca0a3a6b1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:07.798144 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37451b6d-a2ed-44ec-8b04-3ca0a3a6b1e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:07.799446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37451b6d-a2ed-44ec-8b04-3ca0a3a6b1e4 None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.domain.id=default, user.id=1dff148688b84b08aaa5c8689269444f) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:07.799755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37451b6d-a2ed-44ec-8b04-3ca0a3a6b1e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:07.801235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37451b6d-a2ed-44ec-8b04-3ca0a3a6b1e4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:07.808140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 24/53] 10.222.0.86 () {56 vars in 1322 bytes} [Thu Jul 28 17:52:07 2022] GET /identity/v3/role_assignments?user.id=1dff148688b84b08aaa5c8689269444f&scope.domain.id=default&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:07.879523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7c6f-f869-4a36-9f6f-7db54945f187 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:07.879813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7c6f-f869-4a36-9f6f-7db54945f187 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:07.880831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7c6f-f869-4a36-9f6f-7db54945f187 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:07.882935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 30/54] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:07.888737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6b034-03af-448e-a7a7-28f54d728b85 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:07.889069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6b034-03af-448e-a7a7-28f54d728b85 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:07.889360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6b034-03af-448e-a7a7-28f54d728b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:07.908569 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a8c6b034-03af-448e-a7a7-28f54d728b85 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:07.909860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a8c6b034-03af-448e-a7a7-28f54d728b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:07.961931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:07.978613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 25/55] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:08.001590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae7c6f-f869-4a36-9f6f-7db54945f187 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:08.025219 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:08.106634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae7c6f-f869-4a36-9f6f-7db54945f187 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:08.140245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1686dc-8c02-49ba-9e64-e9bc191f8c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:08.140890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1686dc-8c02-49ba-9e64-e9bc191f8c73 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:08.141205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1686dc-8c02-49ba-9e64-e9bc191f8c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:08.141486 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1686dc-8c02-49ba-9e64-e9bc191f8c73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:08.142643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1686dc-8c02-49ba-9e64-e9bc191f8c73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:08.144094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1686dc-8c02-49ba-9e64-e9bc191f8c73 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:08.145724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 31/56] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:08.151544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6b034-03af-448e-a7a7-28f54d728b85 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:08.248176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6b034-03af-448e-a7a7-28f54d728b85 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:08.280075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866412f2-e730-46a7-8060-e6ef26420c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:08.280625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866412f2-e730-46a7-8060-e6ef26420c9a None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:08.280903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866412f2-e730-46a7-8060-e6ef26420c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:08.281215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866412f2-e730-46a7-8060-e6ef26420c9a None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:08.281904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866412f2-e730-46a7-8060-e6ef26420c9a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:08.283811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866412f2-e730-46a7-8060-e6ef26420c9a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:08.362825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 26/57] 10.222.0.86 () {60 vars in 1140 bytes} [Thu Jul 28 17:52:08 2022] POST /identity/v3/projects => generated 298 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:10.344815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d04a7-5099-416e-bf98-7720511d4992 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:10.345875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d04a7-5099-416e-bf98-7720511d4992 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:10.345959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d04a7-5099-416e-bf98-7720511d4992 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:10.347285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 32/58] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:10.352375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7507bd0-9c0c-468b-8f2f-fce62fa0c27a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:10.352795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7507bd0-9c0c-468b-8f2f-fce62fa0c27a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:10.352943 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7507bd0-9c0c-468b-8f2f-fce62fa0c27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:10.371157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f7507bd0-9c0c-468b-8f2f-fce62fa0c27a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:10.372340 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f7507bd0-9c0c-468b-8f2f-fce62fa0c27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:10.399992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:10.437546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 27/59] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:10.461023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d04a7-5099-416e-bf98-7720511d4992 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:10.547626 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:10.560414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d04a7-5099-416e-bf98-7720511d4992 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:10.593558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d28fb-e644-4989-b9e4-6eae7c8692ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:10.594199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d28fb-e644-4989-b9e4-6eae7c8692ce None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:10.594491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d28fb-e644-4989-b9e4-6eae7c8692ce None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:10.594774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d28fb-e644-4989-b9e4-6eae7c8692ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:10.596557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d28fb-e644-4989-b9e4-6eae7c8692ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:10.597979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d28fb-e644-4989-b9e4-6eae7c8692ce None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:10.599554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 33/60] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:10.604891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7507bd0-9c0c-468b-8f2f-fce62fa0c27a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:10.700215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7507bd0-9c0c-468b-8f2f-fce62fa0c27a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:10.733048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce0192-cee8-46f1-b682-0c05b374d56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:10.733659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce0192-cee8-46f1-b682-0c05b374d56e None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:10.733949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce0192-cee8-46f1-b682-0c05b374d56e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:10.734232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce0192-cee8-46f1-b682-0c05b374d56e None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:10.734926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce0192-cee8-46f1-b682-0c05b374d56e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:10.736404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce0192-cee8-46f1-b682-0c05b374d56e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:10.843764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 28/61] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:52:10 2022] POST /identity/v3/users => generated 275 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:52:14.088804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fe811-4136-4bc9-8c36-4ad9333136ce None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.089686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fe811-4136-4bc9-8c36-4ad9333136ce None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.090106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fe811-4136-4bc9-8c36-4ad9333136ce None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.091931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 34/62] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:14.098212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a73c4-d1a1-416c-ac1d-5cd1d8f4d4f2 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.108501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a73c4-d1a1-416c-ac1d-5cd1d8f4d4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.108501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a73c4-d1a1-416c-ac1d-5cd1d8f4d4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.120579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-207a73c4-d1a1-416c-ac1d-5cd1d8f4d4f2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:14.121821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-207a73c4-d1a1-416c-ac1d-5cd1d8f4d4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:14.148569 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:14.218470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 29/63] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:14.221647 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af28c8-ef64-4983-acb0-7e7f2e190b6a None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.222202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af28c8-ef64-4983-acb0-7e7f2e190b6a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.222700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af28c8-ef64-4983-acb0-7e7f2e190b6a None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.224321 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 35/64] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:14.229423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee974f-d6ac-43ff-95f4-0e9e9f907de5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.229724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee974f-d6ac-43ff-95f4-0e9e9f907de5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.230011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee974f-d6ac-43ff-95f4-0e9e9f907de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.249794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af28c8-ef64-4983-acb0-7e7f2e190b6a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:14.254561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-88ee974f-d6ac-43ff-95f4-0e9e9f907de5 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:14.256758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-88ee974f-d6ac-43ff-95f4-0e9e9f907de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:14.352514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 30/65] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:14.354042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450bd3f-ab8e-4cd7-aedc-3da9f36628f0 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.354996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450bd3f-ab8e-4cd7-aedc-3da9f36628f0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.355860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450bd3f-ab8e-4cd7-aedc-3da9f36628f0 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.357500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 31/66] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:14.364463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d9e0e-8a8b-4c40-a5c9-f29905cb288a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.365271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d9e0e-8a8b-4c40-a5c9-f29905cb288a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.365717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d9e0e-8a8b-4c40-a5c9-f29905cb288a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.391233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e27d9e0e-8a8b-4c40-a5c9-f29905cb288a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:14.393206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e27d9e0e-8a8b-4c40-a5c9-f29905cb288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:14.400509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af28c8-ef64-4983-acb0-7e7f2e190b6a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:14.443970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371d0de-4dc0-45df-a2d6-eae8037ec125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:14.444895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371d0de-4dc0-45df-a2d6-eae8037ec125 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.445432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371d0de-4dc0-45df-a2d6-eae8037ec125 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.445918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371d0de-4dc0-45df-a2d6-eae8037ec125 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.467962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371d0de-4dc0-45df-a2d6-eae8037ec125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-9371d0de-4dc0-45df-a2d6-eae8037ec125 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:14.476103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:14.477195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:14.478158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:14.480234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 36/67] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:52:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:14.481403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 32/68] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:14.481589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371d0de-4dc0-45df-a2d6-eae8037ec125 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:14.487030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d9e0e-8a8b-4c40-a5c9-f29905cb288a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:14.508142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:14.515811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:14.596228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371d0de-4dc0-45df-a2d6-eae8037ec125 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:14.608609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d9e0e-8a8b-4c40-a5c9-f29905cb288a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:14.631703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093dbc8-bc6a-4e55-8166-caaf519e4c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:14.632346 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093dbc8-bc6a-4e55-8166-caaf519e4c95 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.632630 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093dbc8-bc6a-4e55-8166-caaf519e4c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.632916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093dbc8-bc6a-4e55-8166-caaf519e4c95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.642809 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093dbc8-bc6a-4e55-8166-caaf519e4c95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:14.643279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64df97-03c6-469f-87a6-299161ec61b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:14.643906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64df97-03c6-469f-87a6-299161ec61b8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.644193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64df97-03c6-469f-87a6-299161ec61b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.644473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64df97-03c6-469f-87a6-299161ec61b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.645222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64df97-03c6-469f-87a6-299161ec61b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:14.645533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64df97-03c6-469f-87a6-299161ec61b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:14.646974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64df97-03c6-469f-87a6-299161ec61b8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-9093dbc8-bc6a-4e55-8166-caaf519e4c95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:14.647723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:14.648667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:14.649457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:14.654203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 37/69] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:14.654439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093dbc8-bc6a-4e55-8166-caaf519e4c95 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:14.656192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 33/70] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:14.657972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63489d-ee68-4a56-92fe-9e0332e1e14a None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.658267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63489d-ee68-4a56-92fe-9e0332e1e14a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.658546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63489d-ee68-4a56-92fe-9e0332e1e14a None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.659771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 34/71] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:14.660799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba53318-f97c-4388-947e-9b4a6553149e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.661166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba53318-f97c-4388-947e-9b4a6553149e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.661446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba53318-f97c-4388-947e-9b4a6553149e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.662954 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 35/72] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:14.666060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba53318-f97c-4388-947e-9b4a6553149e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:14.742374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:14.754184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:14.764398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093dbc8-bc6a-4e55-8166-caaf519e4c95 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:14.778732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba53318-f97c-4388-947e-9b4a6553149e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:14.799481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98959-760e-45a5-ba99-5fd0d1e39c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:14.800132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98959-760e-45a5-ba99-5fd0d1e39c6a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.800424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98959-760e-45a5-ba99-5fd0d1e39c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.800707 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98959-760e-45a5-ba99-5fd0d1e39c6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.811064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc98959-760e-45a5-ba99-5fd0d1e39c6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:14.812822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff4b2d-358b-4cd1-b02a-4a66c334db37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:14.813672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff4b2d-358b-4cd1-b02a-4a66c334db37 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.814119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff4b2d-358b-4cd1-b02a-4a66c334db37 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.814452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff4b2d-358b-4cd1-b02a-4a66c334db37 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.815568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff4b2d-358b-4cd1-b02a-4a66c334db37 None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:14.815883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff4b2d-358b-4cd1-b02a-4a66c334db37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0cc98959-760e-45a5-ba99-5fd0d1e39c6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:14.816047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:14.817064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:14.818113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 38/73] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:52:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:14.818970 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff4b2d-358b-4cd1-b02a-4a66c334db37 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:14.819037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98959-760e-45a5-ba99-5fd0d1e39c6a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:14.830399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 36/74] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:14.831588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf859-ccdd-4ed5-98d3-4e6a5a567a36 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.831878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf859-ccdd-4ed5-98d3-4e6a5a567a36 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.832156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cf859-ccdd-4ed5-98d3-4e6a5a567a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.849798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a92cf859-ccdd-4ed5-98d3-4e6a5a567a36 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:14.850990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a92cf859-ccdd-4ed5-98d3-4e6a5a567a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:14.916508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 37/75] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:14.917777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98959-760e-45a5-ba99-5fd0d1e39c6a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:14.918502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4aa09-c668-459c-9c95-3b7004c61aff None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.918794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4aa09-c668-459c-9c95-3b7004c61aff None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.919076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4aa09-c668-459c-9c95-3b7004c61aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.942005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-55f4aa09-c668-459c-9c95-3b7004c61aff None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:14.943211 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-55f4aa09-c668-459c-9c95-3b7004c61aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:14.952131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf66b5-6f23-4d65-ad40-875d53d98908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:14.952784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf66b5-6f23-4d65-ad40-875d53d98908 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:14.953108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf66b5-6f23-4d65-ad40-875d53d98908 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:14.953394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf66b5-6f23-4d65-ad40-875d53d98908 None admin] PATH_INFO: `/v3/users/1dff148688b84b08aaa5c8689269444f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:14.954614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf66b5-6f23-4d65-ad40-875d53d98908 None admin] RBAC: Authorizing `identity:get_user(user_id=1dff148688b84b08aaa5c8689269444f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:14.956059 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf66b5-6f23-4d65-ad40-875d53d98908 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:14.957709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 39/76] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:14 2022] GET /identity/v3/users/1dff148688b84b08aaa5c8689269444f => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:14.958467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf66b5-6f23-4d65-ad40-875d53d98908 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:14.962517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:15.012370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:15.033017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 38/77] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:15.033017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4aa09-c668-459c-9c95-3b7004c61aff None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:15.073148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf66b5-6f23-4d65-ad40-875d53d98908 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:15.106380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb0f12-5473-48da-a817-fb9d9a883ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:15.107000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb0f12-5473-48da-a817-fb9d9a883ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:15.107282 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb0f12-5473-48da-a817-fb9d9a883ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:15.107567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb0f12-5473-48da-a817-fb9d9a883ab5 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:15.108302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb0f12-5473-48da-a817-fb9d9a883ab5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:15.108615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb0f12-5473-48da-a817-fb9d9a883ab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:15.110040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb0f12-5473-48da-a817-fb9d9a883ab5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:15.117937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 40/78] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:52:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:15.122613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb0f12-5473-48da-a817-fb9d9a883ab5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:15.122704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4aa09-c668-459c-9c95-3b7004c61aff None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:15.156286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa35c40-5348-4270-866c-b73c25f3e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:15.156936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa35c40-5348-4270-866c-b73c25f3e757 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:15.157266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa35c40-5348-4270-866c-b73c25f3e757 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:15.158091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa35c40-5348-4270-866c-b73c25f3e757 None admin] PATH_INFO: `/v3/users/e1f80e64c9e340ae828c1fab2ceed44c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:15.171076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa35c40-5348-4270-866c-b73c25f3e757 None admin] RBAC: Authorizing `identity:get_user(user_id=e1f80e64c9e340ae828c1fab2ceed44c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:15.172469 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa35c40-5348-4270-866c-b73c25f3e757 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:15.174166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 39/79] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:15 2022] GET /identity/v3/users/e1f80e64c9e340ae828c1fab2ceed44c => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:15.178042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa35c40-5348-4270-866c-b73c25f3e757 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:15.184274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:15.219832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb0f12-5473-48da-a817-fb9d9a883ab5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:15.233475 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:15.254031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ecebc-86a3-4689-a049-2e816e42d429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:15.254728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ecebc-86a3-4689-a049-2e816e42d429 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:15.255004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ecebc-86a3-4689-a049-2e816e42d429 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:15.255280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ecebc-86a3-4689-a049-2e816e42d429 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:15.256434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ecebc-86a3-4689-a049-2e816e42d429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:15.257905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ecebc-86a3-4689-a049-2e816e42d429 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:15.259490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 41/80] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:15.260412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ecebc-86a3-4689-a049-2e816e42d429 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:15.279771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa35c40-5348-4270-866c-b73c25f3e757 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:15.313912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d4100-b113-43ed-92fe-7d6bdc4afe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:15.314632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d4100-b113-43ed-92fe-7d6bdc4afe4c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:15.314919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d4100-b113-43ed-92fe-7d6bdc4afe4c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:15.315199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d4100-b113-43ed-92fe-7d6bdc4afe4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:15.358409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ecebc-86a3-4689-a049-2e816e42d429 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:15.382285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d4100-b113-43ed-92fe-7d6bdc4afe4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:15.383739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d4100-b113-43ed-92fe-7d6bdc4afe4c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:15.391327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400a09-a701-4603-a475-fdea0b5d7d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:15.392488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400a09-a701-4603-a475-fdea0b5d7d77 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:15.392772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400a09-a701-4603-a475-fdea0b5d7d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:15.393088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400a09-a701-4603-a475-fdea0b5d7d77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:15.394927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:15.403948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09400a09-a701-4603-a475-fdea0b5d7d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-09400a09-a701-4603-a475-fdea0b5d7d77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:15.409488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:15.410336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:15.411101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 42/81] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:15.411961 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400a09-a701-4603-a475-fdea0b5d7d77 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:15.440179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:15.484869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 40/82] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:15.485765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d4100-b113-43ed-92fe-7d6bdc4afe4c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:15.514613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400a09-a701-4603-a475-fdea0b5d7d77 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:15.548403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fcef1-c52d-42ca-81c4-6a173d6dfcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:15.549083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fcef1-c52d-42ca-81c4-6a173d6dfcba None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:15.549369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fcef1-c52d-42ca-81c4-6a173d6dfcba None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:15.549652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fcef1-c52d-42ca-81c4-6a173d6dfcba None admin] PATH_INFO: `/v3/users/e1f80e64c9e340ae828c1fab2ceed44c` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:15.550851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fcef1-c52d-42ca-81c4-6a173d6dfcba None admin] RBAC: Authorizing `identity:get_user(user_id=e1f80e64c9e340ae828c1fab2ceed44c)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:15.552300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fcef1-c52d-42ca-81c4-6a173d6dfcba None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:15.553948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 43/83] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:15 2022] GET /identity/v3/users/e1f80e64c9e340ae828c1fab2ceed44c => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:15.554695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fcef1-c52d-42ca-81c4-6a173d6dfcba None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:15.589388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d4100-b113-43ed-92fe-7d6bdc4afe4c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:15.615174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:15.622635 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb878cd5-9d5e-4cb9-9545-0ca00d364c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:15.625918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb878cd5-9d5e-4cb9-9545-0ca00d364c2d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:15.625918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb878cd5-9d5e-4cb9-9545-0ca00d364c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:15.625918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb878cd5-9d5e-4cb9-9545-0ca00d364c2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:15.652179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:15.654585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fcef1-c52d-42ca-81c4-6a173d6dfcba None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:15.689118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4355e-50e6-4562-bde9-e86792fe4724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:15.689710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4355e-50e6-4562-bde9-e86792fe4724 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:15.690001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4355e-50e6-4562-bde9-e86792fe4724 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:15.690281 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4355e-50e6-4562-bde9-e86792fe4724 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:15.690976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4355e-50e6-4562-bde9-e86792fe4724 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:15.692475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4355e-50e6-4562-bde9-e86792fe4724 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:15.697222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb878cd5-9d5e-4cb9-9545-0ca00d364c2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:15.698695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb878cd5-9d5e-4cb9-9545-0ca00d364c2d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:15.770406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 44/84] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:52:15 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:15.771038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4355e-50e6-4562-bde9-e86792fe4724 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:15.800445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 41/85] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:15.801298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb878cd5-9d5e-4cb9-9545-0ca00d364c2d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:15.864702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:15.898235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4355e-50e6-4562-bde9-e86792fe4724 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:15.913519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb878cd5-9d5e-4cb9-9545-0ca00d364c2d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:15.918273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:15.939460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1fbe8-d65b-4c0c-a0b6-4ea0b48181e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:15.940796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1fbe8-d65b-4c0c-a0b6-4ea0b48181e1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:15.940796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1fbe8-d65b-4c0c-a0b6-4ea0b48181e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:15.940796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1fbe8-d65b-4c0c-a0b6-4ea0b48181e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:15.941696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1fbe8-d65b-4c0c-a0b6-4ea0b48181e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:15.942019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1fbe8-d65b-4c0c-a0b6-4ea0b48181e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:15.943432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1fbe8-d65b-4c0c-a0b6-4ea0b48181e1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:15.954002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 45/86] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:15.955144 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1fbe8-d65b-4c0c-a0b6-4ea0b48181e1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:15.963826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b65fe0-e564-42af-9293-8dba802e8e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:15.964518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b65fe0-e564-42af-9293-8dba802e8e08 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:15.965208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b65fe0-e564-42af-9293-8dba802e8e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:15.965208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b65fe0-e564-42af-9293-8dba802e8e08 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:15.966301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b65fe0-e564-42af-9293-8dba802e8e08 None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:15.967766 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b65fe0-e564-42af-9293-8dba802e8e08 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:15.969503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 42/87] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:15 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:15.970310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b65fe0-e564-42af-9293-8dba802e8e08 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:16.086394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1fbe8-d65b-4c0c-a0b6-4ea0b48181e1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:16.103888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b65fe0-e564-42af-9293-8dba802e8e08 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:16.136211 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:16.156126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d9bfb-639d-469b-9a81-2a07177595cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:16.156785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d9bfb-639d-469b-9a81-2a07177595cb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:16.157107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d9bfb-639d-469b-9a81-2a07177595cb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:16.157912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d9bfb-639d-469b-9a81-2a07177595cb None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:16.158756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3617f62-a06e-4249-bd7a-7fd2521869be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:16.159214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d9bfb-639d-469b-9a81-2a07177595cb None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:16.160051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3617f62-a06e-4249-bd7a-7fd2521869be None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:16.160543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d9bfb-639d-469b-9a81-2a07177595cb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:16.160727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3617f62-a06e-4249-bd7a-7fd2521869be None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:16.161243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3617f62-a06e-4249-bd7a-7fd2521869be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:16.162279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 43/88] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:15 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:16.163211 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d9bfb-639d-469b-9a81-2a07177595cb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:16.232938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:16.260584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3617f62-a06e-4249-bd7a-7fd2521869be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:16.262967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3617f62-a06e-4249-bd7a-7fd2521869be None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:16.276076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d9bfb-639d-469b-9a81-2a07177595cb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:16.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87477727-d5bf-486e-98a5-154448020bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:16.313420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87477727-d5bf-486e-98a5-154448020bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:16.314181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87477727-d5bf-486e-98a5-154448020bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:16.314467 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87477727-d5bf-486e-98a5-154448020bc4 None admin] PATH_INFO: `/v3/users/e1f80e64c9e340ae828c1fab2ceed44c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:16.315676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87477727-d5bf-486e-98a5-154448020bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=e1f80e64c9e340ae828c1fab2ceed44c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:16.317185 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87477727-d5bf-486e-98a5-154448020bc4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:16.318820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 44/89] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:16 2022] GET /identity/v3/users/e1f80e64c9e340ae828c1fab2ceed44c => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:16.319956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87477727-d5bf-486e-98a5-154448020bc4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:16.396992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:16.403276 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 46/90] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:16.404378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3617f62-a06e-4249-bd7a-7fd2521869be None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:16.450353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87477727-d5bf-486e-98a5-154448020bc4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:16.486270 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f0266-f665-413a-a24d-963555ac9d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:16.486822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f0266-f665-413a-a24d-963555ac9d1b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:16.487110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f0266-f665-413a-a24d-963555ac9d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:16.487398 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f0266-f665-413a-a24d-963555ac9d1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:16.488203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f0266-f665-413a-a24d-963555ac9d1b None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=de970216eec1471ca4c15bf6365e4ef0, user.id=1dff148688b84b08aaa5c8689269444f) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:16.488523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f0266-f665-413a-a24d-963555ac9d1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:16.490093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f0266-f665-413a-a24d-963555ac9d1b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:16.495904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 45/91] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:16 2022] GET /identity/v3/role_assignments?user.id=1dff148688b84b08aaa5c8689269444f&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:16.502790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f0266-f665-413a-a24d-963555ac9d1b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:16.510056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:16.535241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3617f62-a06e-4249-bd7a-7fd2521869be None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:16.579548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1512ea-9db6-418b-ae12-21ffd8a950ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:16.580588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1512ea-9db6-418b-ae12-21ffd8a950ab None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:16.581108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1512ea-9db6-418b-ae12-21ffd8a950ab None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:16.581587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1512ea-9db6-418b-ae12-21ffd8a950ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:16.582611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1512ea-9db6-418b-ae12-21ffd8a950ab None admin] RBAC: Adding query filter params (role.id=2dc5251f14e945e38ad1777a5f235180, scope.project.id=de970216eec1471ca4c15bf6365e4ef0, user.id=e1f80e64c9e340ae828c1fab2ceed44c) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:16.583152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1512ea-9db6-418b-ae12-21ffd8a950ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:16.586240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1512ea-9db6-418b-ae12-21ffd8a950ab None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:16.593410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 47/92] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:16 2022] GET /identity/v3/role_assignments?user.id=e1f80e64c9e340ae828c1fab2ceed44c&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:16.594654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1512ea-9db6-418b-ae12-21ffd8a950ab None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:16.711728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f0266-f665-413a-a24d-963555ac9d1b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:16.724788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1512ea-9db6-418b-ae12-21ffd8a950ab None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:16.749931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:16.780749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743746ac-dabd-481a-bc4f-8b9a3a812634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:16.781917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc11a19-12e0-4b38-a0e8-21a751ec7652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:16.783316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743746ac-dabd-481a-bc4f-8b9a3a812634 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:16.783608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743746ac-dabd-481a-bc4f-8b9a3a812634 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:16.783891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743746ac-dabd-481a-bc4f-8b9a3a812634 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:16.785633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc11a19-12e0-4b38-a0e8-21a751ec7652 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:16.785942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc11a19-12e0-4b38-a0e8-21a751ec7652 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:16.786225 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc11a19-12e0-4b38-a0e8-21a751ec7652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:16.787669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743746ac-dabd-481a-bc4f-8b9a3a812634 None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:16.791538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743746ac-dabd-481a-bc4f-8b9a3a812634 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:16.795908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 48/93] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:16 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:16.799781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743746ac-dabd-481a-bc4f-8b9a3a812634 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:16.831159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:16.885875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc11a19-12e0-4b38-a0e8-21a751ec7652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:16.888611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc11a19-12e0-4b38-a0e8-21a751ec7652 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:16.954820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743746ac-dabd-481a-bc4f-8b9a3a812634 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:16.996353 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073ceb4-fdb9-49c5-8b3c-919c42541493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:16.997558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073ceb4-fdb9-49c5-8b3c-919c42541493 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:16.998051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073ceb4-fdb9-49c5-8b3c-919c42541493 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:16.998558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073ceb4-fdb9-49c5-8b3c-919c42541493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:16.999558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073ceb4-fdb9-49c5-8b3c-919c42541493 None admin] RBAC: Adding query filter params (role.id=f4c7e5a278e34f03a86f1f88ed77db0c, scope.project.id=de970216eec1471ca4c15bf6365e4ef0, user.id=e1f80e64c9e340ae828c1fab2ceed44c) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:17.000127 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073ceb4-fdb9-49c5-8b3c-919c42541493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:17.002989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073ceb4-fdb9-49c5-8b3c-919c42541493 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:17.010912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 49/94] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:16 2022] GET /identity/v3/role_assignments?user.id=e1f80e64c9e340ae828c1fab2ceed44c&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=f4c7e5a278e34f03a86f1f88ed77db0c => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:17.064398 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 46/95] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:17.069814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073ceb4-fdb9-49c5-8b3c-919c42541493 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:17.179042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:17.205556 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073ceb4-fdb9-49c5-8b3c-919c42541493 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:17.252053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b97c9-a393-4068-8640-d2faa0b8539c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:17.253327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b97c9-a393-4068-8640-d2faa0b8539c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:17.253818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b97c9-a393-4068-8640-d2faa0b8539c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:17.254320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b97c9-a393-4068-8640-d2faa0b8539c None admin] PATH_INFO: `/v3/projects/8c15a17276f4486bb1d6c95328e3f6f0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:17.255767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b97c9-a393-4068-8640-d2faa0b8539c None admin] RBAC: Authorizing `identity:get_project(project_id=8c15a17276f4486bb1d6c95328e3f6f0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:17.257515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b97c9-a393-4068-8640-d2faa0b8539c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:17.259894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 50/96] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:17 2022] GET /identity/v3/projects/8c15a17276f4486bb1d6c95328e3f6f0 => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:17.265820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc11a19-12e0-4b38-a0e8-21a751ec7652 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:17.305115 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:17.414461 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc11a19-12e0-4b38-a0e8-21a751ec7652 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:17.462720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4f4d9-cc0f-4495-bc7f-f8f0f6a76e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:17.463356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4f4d9-cc0f-4495-bc7f-f8f0f6a76e80 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:17.463645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4f4d9-cc0f-4495-bc7f-f8f0f6a76e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:17.463929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4f4d9-cc0f-4495-bc7f-f8f0f6a76e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:17.464742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4f4d9-cc0f-4495-bc7f-f8f0f6a76e80 None admin] RBAC: Adding query filter params (role.id=2dc5251f14e945e38ad1777a5f235180, scope.project.id=8c15a17276f4486bb1d6c95328e3f6f0, user.id=e1f80e64c9e340ae828c1fab2ceed44c) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:17.465092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4f4d9-cc0f-4495-bc7f-f8f0f6a76e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:17.466564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4f4d9-cc0f-4495-bc7f-f8f0f6a76e80 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:17.473551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 47/97] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:17 2022] GET /identity/v3/role_assignments?user.id=e1f80e64c9e340ae828c1fab2ceed44c&scope.project.id=8c15a17276f4486bb1d6c95328e3f6f0&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:18.083987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f3f9-454f-4c73-974c-eccd7ec4489e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:18.084683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f3f9-454f-4c73-974c-eccd7ec4489e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:18.085222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f3f9-454f-4c73-974c-eccd7ec4489e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:18.086830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 51/98] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:18.092306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f61b30-34f9-455f-9efd-0727db2059bf None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:18.092602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f61b30-34f9-455f-9efd-0727db2059bf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:18.092888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f61b30-34f9-455f-9efd-0727db2059bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:18.118280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f1f61b30-34f9-455f-9efd-0727db2059bf None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:18.119514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f1f61b30-34f9-455f-9efd-0727db2059bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:18.203899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 48/99] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:18.244276 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64f3f9-454f-4c73-974c-eccd7ec4489e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:18.374197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64f3f9-454f-4c73-974c-eccd7ec4489e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:18.410339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdcfa66-9685-4013-897e-be6d3dcbcf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:18.411378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcfa66-9685-4013-897e-be6d3dcbcf22 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:18.411880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcfa66-9685-4013-897e-be6d3dcbcf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:18.412414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcfa66-9685-4013-897e-be6d3dcbcf22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:18.428008 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcfa66-9685-4013-897e-be6d3dcbcf22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ffdcfa66-9685-4013-897e-be6d3dcbcf22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:18.433458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:18.434296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:18.435055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 52/100] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:18.442130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f61b30-34f9-455f-9efd-0727db2059bf None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:18.472155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:18.591685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f61b30-34f9-455f-9efd-0727db2059bf None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:18.640611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fffd3-2c29-4bda-b321-21fc3c548714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:18.641194 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fffd3-2c29-4bda-b321-21fc3c548714 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:18.641482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fffd3-2c29-4bda-b321-21fc3c548714 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:18.642315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fffd3-2c29-4bda-b321-21fc3c548714 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:18.643047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fffd3-2c29-4bda-b321-21fc3c548714 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:18.643360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fffd3-2c29-4bda-b321-21fc3c548714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:18.644748 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fffd3-2c29-4bda-b321-21fc3c548714 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:18.656510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 49/101] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:18.663741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdcfa66-9685-4013-897e-be6d3dcbcf22 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:18.674588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:18.867428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdcfa66-9685-4013-897e-be6d3dcbcf22 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:18.919671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4fa4f-172f-47ab-ad84-7793def798f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:18.920256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4fa4f-172f-47ab-ad84-7793def798f2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:18.920542 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4fa4f-172f-47ab-ad84-7793def798f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:18.920820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4fa4f-172f-47ab-ad84-7793def798f2 None admin] PATH_INFO: `/v3/users/9c3d93f1023e47a294e3b8fe7a6a387e` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:18.940636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4fa4f-172f-47ab-ad84-7793def798f2 None admin] RBAC: Authorizing `identity:get_user(user_id=9c3d93f1023e47a294e3b8fe7a6a387e)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:18.942153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4fa4f-172f-47ab-ad84-7793def798f2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:18.944188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 53/102] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:18 2022] GET /identity/v3/users/9c3d93f1023e47a294e3b8fe7a6a387e => generated 289 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:18.949577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fffd3-2c29-4bda-b321-21fc3c548714 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:19.066624 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:19.093932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fffd3-2c29-4bda-b321-21fc3c548714 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:19.191076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695bd9d-79a9-438d-bf51-30cfe23e8d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:19.192044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695bd9d-79a9-438d-bf51-30cfe23e8d3e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:19.192536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695bd9d-79a9-438d-bf51-30cfe23e8d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:19.193041 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695bd9d-79a9-438d-bf51-30cfe23e8d3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:19.283588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695bd9d-79a9-438d-bf51-30cfe23e8d3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:19.285463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695bd9d-79a9-438d-bf51-30cfe23e8d3e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:19.328989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2f68d-372b-45c2-a086-b7dd8fa723d1 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:19.328989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2f68d-372b-45c2-a086-b7dd8fa723d1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:19.328989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2f68d-372b-45c2-a086-b7dd8fa723d1 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:19.330933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 54/103] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:19.344707 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca0812-3a52-4cd3-b5df-96c641877005 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:19.345028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca0812-3a52-4cd3-b5df-96c641877005 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:19.345313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca0812-3a52-4cd3-b5df-96c641877005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:19.390010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-44ca0812-3a52-4cd3-b5df-96c641877005 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:19.391258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-44ca0812-3a52-4cd3-b5df-96c641877005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:19.392627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:19.441350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 50/104] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:19.446885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695bd9d-79a9-438d-bf51-30cfe23e8d3e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:19.552108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:19.578317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 55/105] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:19.605102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca0812-3a52-4cd3-b5df-96c641877005 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:19.621685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695bd9d-79a9-438d-bf51-30cfe23e8d3e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:19.684093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b0c62-f760-4515-807d-74287680ff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:19.684760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b0c62-f760-4515-807d-74287680ff77 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:19.685080 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b0c62-f760-4515-807d-74287680ff77 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:19.685489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b0c62-f760-4515-807d-74287680ff77 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:19.686985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b0c62-f760-4515-807d-74287680ff77 None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:19.689044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b0c62-f760-4515-807d-74287680ff77 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:19.690777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 51/106] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:19 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:19.726394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e17b9a-2cf8-4274-8f88-ab3ba83ad611 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:19.726394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e17b9a-2cf8-4274-8f88-ab3ba83ad611 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:19.726394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e17b9a-2cf8-4274-8f88-ab3ba83ad611 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:19.726394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 52/107] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:19 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:19.726394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886eb09-203c-4990-acb3-52fc23ab722f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:19.726394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886eb09-203c-4990-acb3-52fc23ab722f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:19.726875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886eb09-203c-4990-acb3-52fc23ab722f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:19.745593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2886eb09-203c-4990-acb3-52fc23ab722f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:19.747127 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2886eb09-203c-4990-acb3-52fc23ab722f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:19.787558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca0812-3a52-4cd3-b5df-96c641877005 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:19.804130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:19.833308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686f75b-4f64-4659-9965-6f10bb08be30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:19.834851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f75b-4f64-4659-9965-6f10bb08be30 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:19.835324 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f75b-4f64-4659-9965-6f10bb08be30 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:19.835787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f75b-4f64-4659-9965-6f10bb08be30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:19.836632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 53/108] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:19.837509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686f75b-4f64-4659-9965-6f10bb08be30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:19.837856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886eb09-203c-4990-acb3-52fc23ab722f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:19.839421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686f75b-4f64-4659-9965-6f10bb08be30 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:19.842511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 56/109] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:19.843919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4772a2-0b01-49b6-a416-7fbcf39f7227 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:19.844427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4772a2-0b01-49b6-a416-7fbcf39f7227 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:19.844923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4772a2-0b01-49b6-a416-7fbcf39f7227 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:19.846679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 57/110] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:19.849480 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4772a2-0b01-49b6-a416-7fbcf39f7227 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:19.926637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:19.965358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886eb09-203c-4990-acb3-52fc23ab722f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:19.970459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4772a2-0b01-49b6-a416-7fbcf39f7227 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:20.004009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58766c6b-4ee0-4151-b7a7-ebc51a25dab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:20.004937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58766c6b-4ee0-4151-b7a7-ebc51a25dab4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.005546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58766c6b-4ee0-4151-b7a7-ebc51a25dab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.006097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58766c6b-4ee0-4151-b7a7-ebc51a25dab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.007351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58766c6b-4ee0-4151-b7a7-ebc51a25dab4 None admin] RBAC: Adding query filter params (role.id=45dbb625c3064606a023cf5d620476eb, scope.project.id=de970216eec1471ca4c15bf6365e4ef0, user.id=9c3d93f1023e47a294e3b8fe7a6a387e) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:20.007775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58766c6b-4ee0-4151-b7a7-ebc51a25dab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:20.008276 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f3721-cd19-4ccb-8369-497abb00c2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:20.008912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f3721-cd19-4ccb-8369-497abb00c2f2 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.009242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f3721-cd19-4ccb-8369-497abb00c2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.009343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58766c6b-4ee0-4151-b7a7-ebc51a25dab4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:20.009519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f3721-cd19-4ccb-8369-497abb00c2f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.010220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f3721-cd19-4ccb-8369-497abb00c2f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:20.011736 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f3721-cd19-4ccb-8369-497abb00c2f2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:20.015452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 54/111] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:19 2022] GET /identity/v3/role_assignments?user.id=9c3d93f1023e47a294e3b8fe7a6a387e&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=45dbb625c3064606a023cf5d620476eb => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:20.016683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2ca77-016b-4b24-990b-b9c948105ce3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.016992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2ca77-016b-4b24-990b-b9c948105ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.017293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2ca77-016b-4b24-990b-b9c948105ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.043005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a2d2ca77-016b-4b24-990b-b9c948105ce3 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:20.044255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a2d2ca77-016b-4b24-990b-b9c948105ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:20.084314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:20.104882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 58/112] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:52:19 2022] POST /identity/v3/projects => generated 302 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:20.111180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f3721-cd19-4ccb-8369-497abb00c2f2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:20.148905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 55/113] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:20.151433 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a850875-97b5-4c46-947a-ec01c0feca19 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.153488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a850875-97b5-4c46-947a-ec01c0feca19 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.153488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a850875-97b5-4c46-947a-ec01c0feca19 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.155469 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 56/114] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:20.162510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74343ba6-dbc2-4bec-8d49-532283303d9d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.162858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74343ba6-dbc2-4bec-8d49-532283303d9d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.163061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74343ba6-dbc2-4bec-8d49-532283303d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.202080 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-74343ba6-dbc2-4bec-8d49-532283303d9d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:20.203329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-74343ba6-dbc2-4bec-8d49-532283303d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:20.269371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f3721-cd19-4ccb-8369-497abb00c2f2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:20.289431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:20.323880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf23c8-ba6e-4a42-9f3a-9a8d0aaefc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:20.325042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf23c8-ba6e-4a42-9f3a-9a8d0aaefc3a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.325588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf23c8-ba6e-4a42-9f3a-9a8d0aaefc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.326078 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf23c8-ba6e-4a42-9f3a-9a8d0aaefc3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.330479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 57/115] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:20.331702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74343ba6-dbc2-4bec-8d49-532283303d9d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:20.345320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf23c8-ba6e-4a42-9f3a-9a8d0aaefc3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-69bf23c8-ba6e-4a42-9f3a-9a8d0aaefc3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:20.350601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:20.351509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:20.352322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 59/116] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:20.358680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf23c8-ba6e-4a42-9f3a-9a8d0aaefc3a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:20.445891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:20.449896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74343ba6-dbc2-4bec-8d49-532283303d9d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:20.472344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf23c8-ba6e-4a42-9f3a-9a8d0aaefc3a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:20.492695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f6190-c0af-4c15-9237-69ca7503882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:20.493691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6190-c0af-4c15-9237-69ca7503882b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.494160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6190-c0af-4c15-9237-69ca7503882b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.494607 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6190-c0af-4c15-9237-69ca7503882b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.508326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f6190-c0af-4c15-9237-69ca7503882b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:20.510560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fad82d-50c6-4c4f-a2a3-a3cf53d2fb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:20.511195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fad82d-50c6-4c4f-a2a3-a3cf53d2fb3f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.511488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fad82d-50c6-4c4f-a2a3-a3cf53d2fb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.511771 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fad82d-50c6-4c4f-a2a3-a3cf53d2fb3f None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.512521 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fad82d-50c6-4c4f-a2a3-a3cf53d2fb3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:20.512835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fad82d-50c6-4c4f-a2a3-a3cf53d2fb3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:20.514263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fad82d-50c6-4c4f-a2a3-a3cf53d2fb3f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-227f6190-c0af-4c15-9237-69ca7503882b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:20.515724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:20.517124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:20.517895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:20.520373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 58/117] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:52:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:20.521508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f6190-c0af-4c15-9237-69ca7503882b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:20.525049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 60/118] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:20.532054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fad82d-50c6-4c4f-a2a3-a3cf53d2fb3f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:20.573644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:20.633451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f6190-c0af-4c15-9237-69ca7503882b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:20.664323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fad82d-50c6-4c4f-a2a3-a3cf53d2fb3f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:20.672831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85aa28-177b-4e76-93d6-e3f599a08b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:20.673858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85aa28-177b-4e76-93d6-e3f599a08b1e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.674184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85aa28-177b-4e76-93d6-e3f599a08b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.674566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85aa28-177b-4e76-93d6-e3f599a08b1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.686712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85aa28-177b-4e76-93d6-e3f599a08b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ae85aa28-177b-4e76-93d6-e3f599a08b1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:20.692730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:20.693600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:20.694447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 59/119] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:52:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:20.695072 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85aa28-177b-4e76-93d6-e3f599a08b1e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:20.709622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a0b44-eaf9-4125-8a67-dc345cda731a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:20.710284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a0b44-eaf9-4125-8a67-dc345cda731a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.710568 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a0b44-eaf9-4125-8a67-dc345cda731a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.710846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a0b44-eaf9-4125-8a67-dc345cda731a None admin] PATH_INFO: `/v3/users/e1f80e64c9e340ae828c1fab2ceed44c` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.714179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a0b44-eaf9-4125-8a67-dc345cda731a None admin] RBAC: Authorizing `identity:get_user(user_id=e1f80e64c9e340ae828c1fab2ceed44c)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:20.716822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a0b44-eaf9-4125-8a67-dc345cda731a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:20.719908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 61/120] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:20 2022] GET /identity/v3/users/e1f80e64c9e340ae828c1fab2ceed44c => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:20.721550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51503fa4-5d39-42b7-8fdd-5a3d425b14d7 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.722511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51503fa4-5d39-42b7-8fdd-5a3d425b14d7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.723000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51503fa4-5d39-42b7-8fdd-5a3d425b14d7 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.724484 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 62/121] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:20.725771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:20.726834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51503fa4-5d39-42b7-8fdd-5a3d425b14d7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:20.820056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85aa28-177b-4e76-93d6-e3f599a08b1e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:20.851577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51503fa4-5d39-42b7-8fdd-5a3d425b14d7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:20.865333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:20.867020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c2ae3-ac9a-48c0-9d57-9e3f0500a03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:20.874961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c2ae3-ac9a-48c0-9d57-9e3f0500a03c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.874961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c2ae3-ac9a-48c0-9d57-9e3f0500a03c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.874961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c2ae3-ac9a-48c0-9d57-9e3f0500a03c None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.874961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c2ae3-ac9a-48c0-9d57-9e3f0500a03c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:20.874961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c2ae3-ac9a-48c0-9d57-9e3f0500a03c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:20.874961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c2ae3-ac9a-48c0-9d57-9e3f0500a03c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:20.884926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 60/122] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:20.885964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c2ae3-ac9a-48c0-9d57-9e3f0500a03c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:20.894248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86df953-5e67-4f44-b961-1a4b869ed6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:20.895086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df953-5e67-4f44-b961-1a4b869ed6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.895472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df953-5e67-4f44-b961-1a4b869ed6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.895882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df953-5e67-4f44-b961-1a4b869ed6f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.896939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86df953-5e67-4f44-b961-1a4b869ed6f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:20.897291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86df953-5e67-4f44-b961-1a4b869ed6f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:20.903504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86df953-5e67-4f44-b961-1a4b869ed6f6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:20.909400 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 63/123] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:52:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:20.910806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70203c-4676-479c-8e98-ee60a6207b18 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:20.911237 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70203c-4676-479c-8e98-ee60a6207b18 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:20.911607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70203c-4676-479c-8e98-ee60a6207b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:20.932567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3d70203c-4676-479c-8e98-ee60a6207b18 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:20.936280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3d70203c-4676-479c-8e98-ee60a6207b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:20.987449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:21.007322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c2ae3-ac9a-48c0-9d57-9e3f0500a03c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:21.013164 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 64/124] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:21.020054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70203c-4676-479c-8e98-ee60a6207b18 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:21.065747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcd8fb-2da5-41b5-8d4b-d6215416d5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:21.066786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcd8fb-2da5-41b5-8d4b-d6215416d5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:21.067356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcd8fb-2da5-41b5-8d4b-d6215416d5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:21.067880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcd8fb-2da5-41b5-8d4b-d6215416d5c9 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:21.069446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcd8fb-2da5-41b5-8d4b-d6215416d5c9 None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:21.071395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcd8fb-2da5-41b5-8d4b-d6215416d5c9 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:21.073481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 61/125] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:20 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:21.074523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcd8fb-2da5-41b5-8d4b-d6215416d5c9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:21.130628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:21.137781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70203c-4676-479c-8e98-ee60a6207b18 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:21.182746 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2c08e-349c-4172-9748-ae7bcfe71408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:21.183679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2c08e-349c-4172-9748-ae7bcfe71408 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:21.184129 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2c08e-349c-4172-9748-ae7bcfe71408 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:21.184417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2c08e-349c-4172-9748-ae7bcfe71408 None admin] PATH_INFO: `/v3/users/1dff148688b84b08aaa5c8689269444f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:21.185802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2c08e-349c-4172-9748-ae7bcfe71408 None admin] RBAC: Authorizing `identity:get_user(user_id=1dff148688b84b08aaa5c8689269444f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:21.187837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2c08e-349c-4172-9748-ae7bcfe71408 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:21.189981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 65/126] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:21 2022] GET /identity/v3/users/1dff148688b84b08aaa5c8689269444f => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:21.191031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2c08e-349c-4172-9748-ae7bcfe71408 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:21.196144 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcd8fb-2da5-41b5-8d4b-d6215416d5c9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:21.232661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d43b76-15a0-48fa-a89c-c844d0806d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:21.233702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d43b76-15a0-48fa-a89c-c844d0806d74 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:21.234186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d43b76-15a0-48fa-a89c-c844d0806d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:21.234662 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d43b76-15a0-48fa-a89c-c844d0806d74 None admin] PATH_INFO: `/v3/users/e1f80e64c9e340ae828c1fab2ceed44c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:21.236227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d43b76-15a0-48fa-a89c-c844d0806d74 None admin] RBAC: Authorizing `identity:get_user(user_id=e1f80e64c9e340ae828c1fab2ceed44c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:21.238086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d43b76-15a0-48fa-a89c-c844d0806d74 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:21.240090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 62/127] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:21 2022] GET /identity/v3/users/e1f80e64c9e340ae828c1fab2ceed44c => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:21.241302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d43b76-15a0-48fa-a89c-c844d0806d74 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:21.263220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:21.312287 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2c08e-349c-4172-9748-ae7bcfe71408 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:21.348367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5df16-7f65-4efa-8cca-1c09cb7c9090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:21.349031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5df16-7f65-4efa-8cca-1c09cb7c9090 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:21.349315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5df16-7f65-4efa-8cca-1c09cb7c9090 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:21.349589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5df16-7f65-4efa-8cca-1c09cb7c9090 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:21.353465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d43b76-15a0-48fa-a89c-c844d0806d74 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:21.360000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5df16-7f65-4efa-8cca-1c09cb7c9090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-f7c5df16-7f65-4efa-8cca-1c09cb7c9090 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:21.364932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:21.365819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:21.366588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 66/128] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:21.367278 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5df16-7f65-4efa-8cca-1c09cb7c9090 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:21.392283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:21.398845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f998d68-342a-4dd1-87ba-389c5f9ab87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:21.399824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f998d68-342a-4dd1-87ba-389c5f9ab87f None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:21.407856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f998d68-342a-4dd1-87ba-389c5f9ab87f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:21.407856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f998d68-342a-4dd1-87ba-389c5f9ab87f None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0/users/e1f80e64c9e340ae828c1fab2ceed44c/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:21.407856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f998d68-342a-4dd1-87ba-389c5f9ab87f None admin] RBAC: Authorizing `identity:create_grant(project_id=de970216eec1471ca4c15bf6365e4ef0, user_id=e1f80e64c9e340ae828c1fab2ceed44c, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:21.407856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f998d68-342a-4dd1-87ba-389c5f9ab87f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:21.474586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 63/129] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:52:21 2022] PUT /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0/users/e1f80e64c9e340ae828c1fab2ceed44c/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:21.476489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f998d68-342a-4dd1-87ba-389c5f9ab87f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:21.477822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5df16-7f65-4efa-8cca-1c09cb7c9090 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:21.519913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fcb54-2851-4fff-ad3c-821cca0a5e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:21.520713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fcb54-2851-4fff-ad3c-821cca0a5e01 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:21.521221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fcb54-2851-4fff-ad3c-821cca0a5e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:21.521600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fcb54-2851-4fff-ad3c-821cca0a5e01 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:21.523126 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fcb54-2851-4fff-ad3c-821cca0a5e01 None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:21.524774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fcb54-2851-4fff-ad3c-821cca0a5e01 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:21.526771 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 67/130] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:21 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:21.528007 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fcb54-2851-4fff-ad3c-821cca0a5e01 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:21.590431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:21.621869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f998d68-342a-4dd1-87ba-389c5f9ab87f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:21.634009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:21.645877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fcb54-2851-4fff-ad3c-821cca0a5e01 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:21.671778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17653282-4199-4823-8a15-9dad7bbfc09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:21.673587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17653282-4199-4823-8a15-9dad7bbfc09e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:21.674337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17653282-4199-4823-8a15-9dad7bbfc09e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:21.674772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17653282-4199-4823-8a15-9dad7bbfc09e None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:21.676589 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17653282-4199-4823-8a15-9dad7bbfc09e None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:21.679018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17653282-4199-4823-8a15-9dad7bbfc09e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:21.682079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 64/131] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:21 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:21.683335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17653282-4199-4823-8a15-9dad7bbfc09e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:21.696506 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04604db1-ccf9-459e-bb7b-065db954ac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:21.697366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04604db1-ccf9-459e-bb7b-065db954ac68 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:21.697749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04604db1-ccf9-459e-bb7b-065db954ac68 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:21.698489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04604db1-ccf9-459e-bb7b-065db954ac68 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:21.699056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04604db1-ccf9-459e-bb7b-065db954ac68 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:21.699508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04604db1-ccf9-459e-bb7b-065db954ac68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:21.701206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04604db1-ccf9-459e-bb7b-065db954ac68 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:21.721673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 68/132] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:21.722980 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04604db1-ccf9-459e-bb7b-065db954ac68 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:21.814137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17653282-4199-4823-8a15-9dad7bbfc09e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:21.858478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab76ea-5958-4869-9f4c-284dfdf355a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:21.859807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab76ea-5958-4869-9f4c-284dfdf355a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:21.860760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab76ea-5958-4869-9f4c-284dfdf355a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:21.861306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab76ea-5958-4869-9f4c-284dfdf355a3 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0/users/1dff148688b84b08aaa5c8689269444f/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:21.864094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab76ea-5958-4869-9f4c-284dfdf355a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=de970216eec1471ca4c15bf6365e4ef0, user_id=1dff148688b84b08aaa5c8689269444f, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:21.866082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab76ea-5958-4869-9f4c-284dfdf355a3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:21.884936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04604db1-ccf9-459e-bb7b-065db954ac68 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:21.921260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fcb28-0b6f-4f56-a4a7-4f047f2fd343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:21.921927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fcb28-0b6f-4f56-a4a7-4f047f2fd343 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:21.922216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fcb28-0b6f-4f56-a4a7-4f047f2fd343 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:21.922618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fcb28-0b6f-4f56-a4a7-4f047f2fd343 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0/users/e1f80e64c9e340ae828c1fab2ceed44c/roles/f4c7e5a278e34f03a86f1f88ed77db0c` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:21.925412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fcb28-0b6f-4f56-a4a7-4f047f2fd343 None admin] RBAC: Authorizing `identity:create_grant(project_id=de970216eec1471ca4c15bf6365e4ef0, user_id=e1f80e64c9e340ae828c1fab2ceed44c, role_id=f4c7e5a278e34f03a86f1f88ed77db0c)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:21.926932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fcb28-0b6f-4f56-a4a7-4f047f2fd343 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:21.976352 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 65/133] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:52:21 2022] PUT /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0/users/1dff148688b84b08aaa5c8689269444f/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:21.977108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab76ea-5958-4869-9f4c-284dfdf355a3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:21.995542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:22.062968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 69/134] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:52:21 2022] PUT /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0/users/e1f80e64c9e340ae828c1fab2ceed44c/roles/f4c7e5a278e34f03a86f1f88ed77db0c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:22.142038 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab76ea-5958-4869-9f4c-284dfdf355a3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:22.183002 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b6898-9be0-4754-b06b-910172ae4ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:22.183691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b6898-9be0-4754-b06b-910172ae4ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:22.183980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b6898-9be0-4754-b06b-910172ae4ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:22.184265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b6898-9be0-4754-b06b-910172ae4ab3 None admin] PATH_INFO: `/v3/users/e1f80e64c9e340ae828c1fab2ceed44c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:22.185573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b6898-9be0-4754-b06b-910172ae4ab3 None admin] RBAC: Authorizing `identity:get_user(user_id=e1f80e64c9e340ae828c1fab2ceed44c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:22.187081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b6898-9be0-4754-b06b-910172ae4ab3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:22.189329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 66/135] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:21 2022] GET /identity/v3/users/e1f80e64c9e340ae828c1fab2ceed44c => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:22.195296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fcb28-0b6f-4f56-a4a7-4f047f2fd343 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:22.200523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477230e6-ed43-4c06-a851-35fdb066a308 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:22.200820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477230e6-ed43-4c06-a851-35fdb066a308 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:22.201140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477230e6-ed43-4c06-a851-35fdb066a308 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:22.202550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 67/136] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:52:22.209349 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40841b47-6f3a-4999-a067-243d7db2ea7c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:22.209744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40841b47-6f3a-4999-a067-243d7db2ea7c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:22.210077 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40841b47-6f3a-4999-a067-243d7db2ea7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:22.218596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:22.229261 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-40841b47-6f3a-4999-a067-243d7db2ea7c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:22.233746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-40841b47-6f3a-4999-a067-243d7db2ea7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:22.341621 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 68/137] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:22.361995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fcb28-0b6f-4f56-a4a7-4f047f2fd343 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:22.384895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40841b47-6f3a-4999-a067-243d7db2ea7c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:22.394296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:22.404638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c1e17-98e5-4858-a468-dd5846fcc134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:22.405931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c1e17-98e5-4858-a468-dd5846fcc134 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:22.405931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c1e17-98e5-4858-a468-dd5846fcc134 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:22.405931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c1e17-98e5-4858-a468-dd5846fcc134 None admin] PATH_INFO: `/v3/projects/8c15a17276f4486bb1d6c95328e3f6f0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:22.417038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c1e17-98e5-4858-a468-dd5846fcc134 None admin] RBAC: Authorizing `identity:get_project(project_id=8c15a17276f4486bb1d6c95328e3f6f0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:22.417038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c1e17-98e5-4858-a468-dd5846fcc134 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:22.417038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 70/138] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:22 2022] GET /identity/v3/projects/8c15a17276f4486bb1d6c95328e3f6f0 => generated 312 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:22.417038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c1e17-98e5-4858-a468-dd5846fcc134 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:22.524949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40841b47-6f3a-4999-a067-243d7db2ea7c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:22.579429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:22.593899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0beb90-89f7-45ad-8987-aab35c95d5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:22.594792 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0beb90-89f7-45ad-8987-aab35c95d5cf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:22.594792 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0beb90-89f7-45ad-8987-aab35c95d5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:22.596415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0beb90-89f7-45ad-8987-aab35c95d5cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:22.600152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c1e17-98e5-4858-a468-dd5846fcc134 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:22.629990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0beb90-89f7-45ad-8987-aab35c95d5cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-1f0beb90-89f7-45ad-8987-aab35c95d5cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:22.635069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:22.636150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:22.637093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 69/139] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:22.646089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0beb90-89f7-45ad-8987-aab35c95d5cf None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:22.648988 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35831b99-d575-43f4-997d-9ea3e6fffd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:22.649860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35831b99-d575-43f4-997d-9ea3e6fffd53 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:22.650293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35831b99-d575-43f4-997d-9ea3e6fffd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:22.650738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35831b99-d575-43f4-997d-9ea3e6fffd53 None admin] PATH_INFO: `/v3/projects/8c15a17276f4486bb1d6c95328e3f6f0/users/e1f80e64c9e340ae828c1fab2ceed44c/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:22.658433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35831b99-d575-43f4-997d-9ea3e6fffd53 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c15a17276f4486bb1d6c95328e3f6f0, user_id=e1f80e64c9e340ae828c1fab2ceed44c, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:22.660061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35831b99-d575-43f4-997d-9ea3e6fffd53 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:22.741868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 71/140] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:52:22 2022] PUT /identity/v3/projects/8c15a17276f4486bb1d6c95328e3f6f0/users/e1f80e64c9e340ae828c1fab2ceed44c/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:52:22.745089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:22.794918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0beb90-89f7-45ad-8987-aab35c95d5cf None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:22.841943 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e12876-aa0b-4767-96c6-065d7fd2bfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:22.842600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e12876-aa0b-4767-96c6-065d7fd2bfb6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:22.842888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e12876-aa0b-4767-96c6-065d7fd2bfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:22.843168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e12876-aa0b-4767-96c6-065d7fd2bfb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:22.843924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e12876-aa0b-4767-96c6-065d7fd2bfb6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:22.844238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e12876-aa0b-4767-96c6-065d7fd2bfb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:22.845725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e12876-aa0b-4767-96c6-065d7fd2bfb6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:22.859554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 70/141] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:22.866840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35831b99-d575-43f4-997d-9ea3e6fffd53 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:22.928992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdb50a-755a-462a-9971-0394d4b08498 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:22.929512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdb50a-755a-462a-9971-0394d4b08498 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:22.929963 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdb50a-755a-462a-9971-0394d4b08498 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:22.931470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 71/142] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:22.937982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9773-fb6d-4450-b2e2-35cd5c9af23d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:22.938446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9773-fb6d-4450-b2e2-35cd5c9af23d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:22.938885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9773-fb6d-4450-b2e2-35cd5c9af23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:22.960086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4c5b9773-fb6d-4450-b2e2-35cd5c9af23d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:22.962059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4c5b9773-fb6d-4450-b2e2-35cd5c9af23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:23.055137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35831b99-d575-43f4-997d-9ea3e6fffd53 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:23.080643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 72/143] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:23.111434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ffcdf4-4929-4641-aeec-18995d4fe94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:23.112322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffcdf4-4929-4641-aeec-18995d4fe94d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:23.112742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffcdf4-4929-4641-aeec-18995d4fe94d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:23.113233 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffcdf4-4929-4641-aeec-18995d4fe94d None admin] PATH_INFO: `/v3/users/9c3d93f1023e47a294e3b8fe7a6a387e` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:23.114781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ffcdf4-4929-4641-aeec-18995d4fe94d None admin] RBAC: Authorizing `identity:get_user(user_id=9c3d93f1023e47a294e3b8fe7a6a387e)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:23.116489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ffcdf4-4929-4641-aeec-18995d4fe94d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:23.118336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 72/144] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:22 2022] GET /identity/v3/users/9c3d93f1023e47a294e3b8fe7a6a387e => generated 289 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:23.124147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b9773-fb6d-4450-b2e2-35cd5c9af23d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:23.186429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:23.197992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ffcdf4-4929-4641-aeec-18995d4fe94d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:23.218019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:23.283863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b9773-fb6d-4450-b2e2-35cd5c9af23d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:23.347032 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772e2a8-6fab-4805-a10d-2a7d9a1df5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:23.347904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772e2a8-6fab-4805-a10d-2a7d9a1df5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:23.348223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772e2a8-6fab-4805-a10d-2a7d9a1df5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:23.348601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772e2a8-6fab-4805-a10d-2a7d9a1df5e8 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:23.353692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772e2a8-6fab-4805-a10d-2a7d9a1df5e8 None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:23.355251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772e2a8-6fab-4805-a10d-2a7d9a1df5e8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:23.357434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 73/145] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:23 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:23.363848 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772e2a8-6fab-4805-a10d-2a7d9a1df5e8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:23.365324 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ffcdf4-4929-4641-aeec-18995d4fe94d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:23.404911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0aec7-9636-4ac5-80d4-05d4dcc21b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:23.406200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0aec7-9636-4ac5-80d4-05d4dcc21b4a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:23.406696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0aec7-9636-4ac5-80d4-05d4dcc21b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:23.407831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0aec7-9636-4ac5-80d4-05d4dcc21b4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:23.409229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0aec7-9636-4ac5-80d4-05d4dcc21b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:23.410800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0aec7-9636-4ac5-80d4-05d4dcc21b4a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:23.412684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 73/146] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:23.418599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0aec7-9636-4ac5-80d4-05d4dcc21b4a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:23.504299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:23.520697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772e2a8-6fab-4805-a10d-2a7d9a1df5e8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:23.530415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0aec7-9636-4ac5-80d4-05d4dcc21b4a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:23.567039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:23.570451 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7afc2-2aa4-41a7-9ab0-069c0fc32945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:23.572392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7afc2-2aa4-41a7-9ab0-069c0fc32945 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:23.573543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7afc2-2aa4-41a7-9ab0-069c0fc32945 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:23.574702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7afc2-2aa4-41a7-9ab0-069c0fc32945 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:23.575536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7afc2-2aa4-41a7-9ab0-069c0fc32945 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:23.579590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7afc2-2aa4-41a7-9ab0-069c0fc32945 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:23.592266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b0e6c-140b-40a7-a954-9d57b160d170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:23.593246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b0e6c-140b-40a7-a954-9d57b160d170 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:23.593700 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b0e6c-140b-40a7-a954-9d57b160d170 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:23.594139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b0e6c-140b-40a7-a954-9d57b160d170 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0/users/9c3d93f1023e47a294e3b8fe7a6a387e/roles/45dbb625c3064606a023cf5d620476eb` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:23.598201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b0e6c-140b-40a7-a954-9d57b160d170 None admin] RBAC: Authorizing `identity:create_grant(project_id=de970216eec1471ca4c15bf6365e4ef0, user_id=9c3d93f1023e47a294e3b8fe7a6a387e, role_id=45dbb625c3064606a023cf5d620476eb)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:23.601782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b0e6c-140b-40a7-a954-9d57b160d170 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:23.713780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 74/147] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:52:23 2022] POST /identity/v3/users => generated 283 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:52:23.753266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 74/148] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:52:23 2022] PUT /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0/users/9c3d93f1023e47a294e3b8fe7a6a387e/roles/45dbb625c3064606a023cf5d620476eb => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:52:24.583493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751acb2c-9d78-4314-87f2-feed29f04240 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:24.584501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751acb2c-9d78-4314-87f2-feed29f04240 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:24.585044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751acb2c-9d78-4314-87f2-feed29f04240 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:24.586621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 75/149] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:24.592282 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2ecdd-0ee1-43ac-8727-806dafa458c8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:24.593200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2ecdd-0ee1-43ac-8727-806dafa458c8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:24.593710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2ecdd-0ee1-43ac-8727-806dafa458c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:24.617712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d9b2ecdd-0ee1-43ac-8727-806dafa458c8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:24.619398 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d9b2ecdd-0ee1-43ac-8727-806dafa458c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:24.703757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 75/150] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:24.742937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751acb2c-9d78-4314-87f2-feed29f04240 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:24.862732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751acb2c-9d78-4314-87f2-feed29f04240 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:24.900148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:24.909408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d323ad8-434b-4631-995f-cf463b6bac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:24.910396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d323ad8-434b-4631-995f-cf463b6bac87 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:24.910839 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d323ad8-434b-4631-995f-cf463b6bac87 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:24.911277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d323ad8-434b-4631-995f-cf463b6bac87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:24.926638 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65054-63f3-44ab-b754-ceb09c446978 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:24.927251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65054-63f3-44ab-b754-ceb09c446978 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:24.927723 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65054-63f3-44ab-b754-ceb09c446978 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:24.929462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 76/151] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:52:24.931870 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d323ad8-434b-4631-995f-cf463b6bac87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:24.934492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60fa70-074e-407d-87a9-9fdd8e63f780 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:24.934999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60fa70-074e-407d-87a9-9fdd8e63f780 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:24.935523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60fa70-074e-407d-87a9-9fdd8e63f780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-7d323ad8-434b-4631-995f-cf463b6bac87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:24.953045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:24.954625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:24.955394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:24.958695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 76/152] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:24.965207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d323ad8-434b-4631-995f-cf463b6bac87 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:24.973722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ec60fa70-074e-407d-87a9-9fdd8e63f780 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:24.975294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ec60fa70-074e-407d-87a9-9fdd8e63f780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:25.130952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d323ad8-434b-4631-995f-cf463b6bac87 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:25.180802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a49a8-149c-4a7f-a8ae-006589beddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:25.181865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a49a8-149c-4a7f-a8ae-006589beddab None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.182570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a49a8-149c-4a7f-a8ae-006589beddab None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.183042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a49a8-149c-4a7f-a8ae-006589beddab None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:25.184522 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a49a8-149c-4a7f-a8ae-006589beddab None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:25.185226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a49a8-149c-4a7f-a8ae-006589beddab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:25.186836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a49a8-149c-4a7f-a8ae-006589beddab None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:25.193409 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 77/153] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:25.197420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 77/154] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:25.203149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60fa70-074e-407d-87a9-9fdd8e63f780 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:25.233258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a49a8-149c-4a7f-a8ae-006589beddab None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:25.305709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:25.315033 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:25.380784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60fa70-074e-407d-87a9-9fdd8e63f780 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:25.382857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a49a8-149c-4a7f-a8ae-006589beddab None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:25.426298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4aaa50-3e10-49af-acc5-356440240ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:25.426979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4aaa50-3e10-49af-acc5-356440240ebf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.427263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4aaa50-3e10-49af-acc5-356440240ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.427541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4aaa50-3e10-49af-acc5-356440240ebf None admin] PATH_INFO: `/v3/users/e1f80e64c9e340ae828c1fab2ceed44c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:25.428749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4aaa50-3e10-49af-acc5-356440240ebf None admin] RBAC: Authorizing `identity:get_user(user_id=e1f80e64c9e340ae828c1fab2ceed44c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:25.430565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4aaa50-3e10-49af-acc5-356440240ebf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:25.433008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 78/155] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:25 2022] GET /identity/v3/users/e1f80e64c9e340ae828c1fab2ceed44c => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:25.434025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce949e94-1f45-4878-9797-7460d0c9a0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:25.434948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce949e94-1f45-4878-9797-7460d0c9a0ef None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.435303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce949e94-1f45-4878-9797-7460d0c9a0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.435653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce949e94-1f45-4878-9797-7460d0c9a0ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:25.437917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4aaa50-3e10-49af-acc5-356440240ebf None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:25.453665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce949e94-1f45-4878-9797-7460d0c9a0ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ce949e94-1f45-4878-9797-7460d0c9a0ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:25.459832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:25.460716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:25.461583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:25.470221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 78/156] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:52:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:25.476196 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce949e94-1f45-4878-9797-7460d0c9a0ef None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:25.594896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4aaa50-3e10-49af-acc5-356440240ebf None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:25.619040 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:25.638117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce949e94-1f45-4878-9797-7460d0c9a0ef None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:25.648571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779fc16-a9c3-44ab-810e-c8f31b08e7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:25.649243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779fc16-a9c3-44ab-810e-c8f31b08e7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.649532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779fc16-a9c3-44ab-810e-c8f31b08e7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.649812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779fc16-a9c3-44ab-810e-c8f31b08e7e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:25.658499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:25.682638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7af5b-0b87-4369-b66f-535d508a2f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:25.683581 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7af5b-0b87-4369-b66f-535d508a2f12 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.683869 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7af5b-0b87-4369-b66f-535d508a2f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.684151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7af5b-0b87-4369-b66f-535d508a2f12 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:25.687655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7af5b-0b87-4369-b66f-535d508a2f12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:25.687973 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7af5b-0b87-4369-b66f-535d508a2f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:25.691017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7af5b-0b87-4369-b66f-535d508a2f12 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:25.704624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 79/157] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:52:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:25.707805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccd13f-4afa-4c29-9aae-f92a63a0bb5b None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.708093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccd13f-4afa-4c29-9aae-f92a63a0bb5b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.708371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccd13f-4afa-4c29-9aae-f92a63a0bb5b None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:25.710559 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 80/158] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:25.717864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccd13f-4afa-4c29-9aae-f92a63a0bb5b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:25.787511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779fc16-a9c3-44ab-810e-c8f31b08e7e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:25.789034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779fc16-a9c3-44ab-810e-c8f31b08e7e0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:25.936696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccd13f-4afa-4c29-9aae-f92a63a0bb5b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:25.988733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594b424-fc11-454a-b585-084b57dd927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:25.989429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594b424-fc11-454a-b585-084b57dd927d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.989712 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594b424-fc11-454a-b585-084b57dd927d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.991444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594b424-fc11-454a-b585-084b57dd927d None admin] PATH_INFO: `/v3/users/e1f80e64c9e340ae828c1fab2ceed44c` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:25.992865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594b424-fc11-454a-b585-084b57dd927d None admin] RBAC: Authorizing `identity:get_user(user_id=e1f80e64c9e340ae828c1fab2ceed44c)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:25.994444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594b424-fc11-454a-b585-084b57dd927d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:25.996239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 81/159] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:25 2022] GET /identity/v3/users/e1f80e64c9e340ae828c1fab2ceed44c => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:25.997557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e030f-a6e1-4b07-a34c-32f86f7192a2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.997956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e030f-a6e1-4b07-a34c-32f86f7192a2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.998238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e030f-a6e1-4b07-a34c-32f86f7192a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.011897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 79/160] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:26.012837 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779fc16-a9c3-44ab-810e-c8f31b08e7e0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.032507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:26.040880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-cb0e030f-a6e1-4b07-a34c-32f86f7192a2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:26.042117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cb0e030f-a6e1-4b07-a34c-32f86f7192a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:26.161807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:26.176381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779fc16-a9c3-44ab-810e-c8f31b08e7e0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:26.227717 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 82/161] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:26.229039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e030f-a6e1-4b07-a34c-32f86f7192a2 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.234276 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3c204-deac-4af4-afe3-305f8a26a072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:26.234923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3c204-deac-4af4-afe3-305f8a26a072 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.235772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3c204-deac-4af4-afe3-305f8a26a072 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.236060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3c204-deac-4af4-afe3-305f8a26a072 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.348841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3c204-deac-4af4-afe3-305f8a26a072 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:26.350790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3c204-deac-4af4-afe3-305f8a26a072 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:26.372561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:26.455634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e030f-a6e1-4b07-a34c-32f86f7192a2 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:26.552212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46834deb-6313-4f71-955c-01063cbd5928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:26.552794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46834deb-6313-4f71-955c-01063cbd5928 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.553104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46834deb-6313-4f71-955c-01063cbd5928 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.553380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46834deb-6313-4f71-955c-01063cbd5928 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.554594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46834deb-6313-4f71-955c-01063cbd5928 None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:26.556043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46834deb-6313-4f71-955c-01063cbd5928 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:26.557755 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 83/162] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:26 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:26.558508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46834deb-6313-4f71-955c-01063cbd5928 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.575171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 80/163] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:26.576468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439ece5-f256-4321-990d-ac5000e4b547 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.576752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439ece5-f256-4321-990d-ac5000e4b547 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.577051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439ece5-f256-4321-990d-ac5000e4b547 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.578318 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 81/164] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:26.584090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439ece5-f256-4321-990d-ac5000e4b547 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.667298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:26.777405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46834deb-6313-4f71-955c-01063cbd5928 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:26.845876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439ece5-f256-4321-990d-ac5000e4b547 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:26.854272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487b88f-cb80-478a-8b1b-ab70525312c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:26.854934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487b88f-cb80-478a-8b1b-ab70525312c7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.855217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487b88f-cb80-478a-8b1b-ab70525312c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.855497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487b88f-cb80-478a-8b1b-ab70525312c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.869248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:26.877667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487b88f-cb80-478a-8b1b-ab70525312c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a487b88f-cb80-478a-8b1b-ab70525312c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:26.883720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:26.885971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:26.886760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 84/165] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:52:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:26.889068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487b88f-cb80-478a-8b1b-ab70525312c7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.906920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdec059-12a5-4146-85d4-8ff2a999cdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:26.907596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec059-12a5-4146-85d4-8ff2a999cdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.907886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec059-12a5-4146-85d4-8ff2a999cdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.908172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec059-12a5-4146-85d4-8ff2a999cdd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.909030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdec059-12a5-4146-85d4-8ff2a999cdd4 None admin] RBAC: Adding query filter params (role.id=2dc5251f14e945e38ad1777a5f235180, scope.project.id=de970216eec1471ca4c15bf6365e4ef0, user.id=e1f80e64c9e340ae828c1fab2ceed44c) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:26.909358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdec059-12a5-4146-85d4-8ff2a999cdd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:26.910824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdec059-12a5-4146-85d4-8ff2a999cdd4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:26.922382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 82/166] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:26 2022] GET /identity/v3/role_assignments?user.id=e1f80e64c9e340ae828c1fab2ceed44c&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 608 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:26.923704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e64fb-bf9f-4b9a-b796-5e38a93f64e1 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.924029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e64fb-bf9f-4b9a-b796-5e38a93f64e1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.924309 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e64fb-bf9f-4b9a-b796-5e38a93f64e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.994270 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2c0e64fb-bf9f-4b9a-b796-5e38a93f64e1 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:26.995697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2c0e64fb-bf9f-4b9a-b796-5e38a93f64e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:27.108050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487b88f-cb80-478a-8b1b-ab70525312c7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:27.121356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:27.141797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 83/167] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:27.144335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d97ddf-0695-4ee5-8927-7e0f6c1c11c4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:27.144637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d97ddf-0695-4ee5-8927-7e0f6c1c11c4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:27.144921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d97ddf-0695-4ee5-8927-7e0f6c1c11c4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:27.148249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 84/168] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:27.149003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d97ddf-0695-4ee5-8927-7e0f6c1c11c4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:27.179521 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebbbe4-6678-4504-a0f7-fece33b9ffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:27.181222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebbbe4-6678-4504-a0f7-fece33b9ffe0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:27.181527 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebbbe4-6678-4504-a0f7-fece33b9ffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:27.181810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebbbe4-6678-4504-a0f7-fece33b9ffe0 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:27.183858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebbbe4-6678-4504-a0f7-fece33b9ffe0 None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:27.185470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebbbe4-6678-4504-a0f7-fece33b9ffe0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:27.187678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 85/169] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:26 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:27.189304 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf898f-fa32-4001-9d6f-f65bdfd823cd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:27.189591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf898f-fa32-4001-9d6f-f65bdfd823cd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:27.189866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf898f-fa32-4001-9d6f-f65bdfd823cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:27.215495 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a6bf898f-fa32-4001-9d6f-f65bdfd823cd None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:27.216755 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a6bf898f-fa32-4001-9d6f-f65bdfd823cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:27.334772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 86/170] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:27.335691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf898f-fa32-4001-9d6f-f65bdfd823cd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:27.417778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d97ddf-0695-4ee5-8927-7e0f6c1c11c4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:27.430277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:27.484272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf898f-fa32-4001-9d6f-f65bdfd823cd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:27.497363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:27.518289 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878b14d-183a-4130-9260-67fad60e4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:27.518944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878b14d-183a-4130-9260-67fad60e4963 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:27.519229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878b14d-183a-4130-9260-67fad60e4963 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:27.519510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878b14d-183a-4130-9260-67fad60e4963 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:27.520362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878b14d-183a-4130-9260-67fad60e4963 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:27.520676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878b14d-183a-4130-9260-67fad60e4963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:27.522109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878b14d-183a-4130-9260-67fad60e4963 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:27.529146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b2848-1b8b-43ea-b3b1-d2a18f9a8f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:27.530350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b2848-1b8b-43ea-b3b1-d2a18f9a8f6a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:27.530632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b2848-1b8b-43ea-b3b1-d2a18f9a8f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:27.531054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b2848-1b8b-43ea-b3b1-d2a18f9a8f6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:27.533090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 85/171] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:27.534034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878b14d-183a-4130-9260-67fad60e4963 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:27.550309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b2848-1b8b-43ea-b3b1-d2a18f9a8f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-787b2848-1b8b-43ea-b3b1-d2a18f9a8f6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:27.556192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:27.557108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:27.557907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 87/172] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:27.558559 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b2848-1b8b-43ea-b3b1-d2a18f9a8f6a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:27.721588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878b14d-183a-4130-9260-67fad60e4963 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:27.732419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b2848-1b8b-43ea-b3b1-d2a18f9a8f6a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:27.759152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05959d5-16a3-4827-ac71-5216feadbd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:27.759805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05959d5-16a3-4827-ac71-5216feadbd31 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:27.760088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05959d5-16a3-4827-ac71-5216feadbd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:27.760366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05959d5-16a3-4827-ac71-5216feadbd31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:27.761343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05959d5-16a3-4827-ac71-5216feadbd31 None admin] RBAC: Adding query filter params (role.id=f4c7e5a278e34f03a86f1f88ed77db0c, scope.project.id=de970216eec1471ca4c15bf6365e4ef0, user.id=e1f80e64c9e340ae828c1fab2ceed44c) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:27.761738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05959d5-16a3-4827-ac71-5216feadbd31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:27.763198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05959d5-16a3-4827-ac71-5216feadbd31 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:27.769925 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 86/173] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:27 2022] GET /identity/v3/role_assignments?user.id=e1f80e64c9e340ae828c1fab2ceed44c&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=f4c7e5a278e34f03a86f1f88ed77db0c => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:27.770732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05959d5-16a3-4827-ac71-5216feadbd31 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:27.785805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fabf6e-1277-40fe-8d57-0fb1a41a3a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:27.787171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fabf6e-1277-40fe-8d57-0fb1a41a3a15 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:27.787459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fabf6e-1277-40fe-8d57-0fb1a41a3a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:27.787736 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fabf6e-1277-40fe-8d57-0fb1a41a3a15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:27.817275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fabf6e-1277-40fe-8d57-0fb1a41a3a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-07fabf6e-1277-40fe-8d57-0fb1a41a3a15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:27.825120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:27.826086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:27.827307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 88/174] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:27.831487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fabf6e-1277-40fe-8d57-0fb1a41a3a15 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:27.840212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:27.883778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:27.980906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05959d5-16a3-4827-ac71-5216feadbd31 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:28.005692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fabf6e-1277-40fe-8d57-0fb1a41a3a15 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:28.026516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8ea22-f4f0-4812-808a-9f4cb1b6ed8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:28.027169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ea22-f4f0-4812-808a-9f4cb1b6ed8d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.027547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ea22-f4f0-4812-808a-9f4cb1b6ed8d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.027827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ea22-f4f0-4812-808a-9f4cb1b6ed8d None admin] PATH_INFO: `/v3/users/1dff148688b84b08aaa5c8689269444f` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.029041 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8ea22-f4f0-4812-808a-9f4cb1b6ed8d None admin] RBAC: Authorizing `identity:get_user(user_id=1dff148688b84b08aaa5c8689269444f)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:28.031334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8ea22-f4f0-4812-808a-9f4cb1b6ed8d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:28.038743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 87/175] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:27 2022] GET /identity/v3/users/1dff148688b84b08aaa5c8689269444f => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:28.038743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a80b3-c43a-466b-ac89-9e1ea9198d4b None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.038743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a80b3-c43a-466b-ac89-9e1ea9198d4b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.038743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a80b3-c43a-466b-ac89-9e1ea9198d4b None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.038743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 88/176] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:28.038743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a80b3-c43a-466b-ac89-9e1ea9198d4b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:28.055879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c254e8-b97d-4809-8580-5dffbab52aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:28.056808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c254e8-b97d-4809-8580-5dffbab52aca None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.057305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c254e8-b97d-4809-8580-5dffbab52aca None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.057821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c254e8-b97d-4809-8580-5dffbab52aca None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.058735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c254e8-b97d-4809-8580-5dffbab52aca None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:28.059232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c254e8-b97d-4809-8580-5dffbab52aca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:28.060873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c254e8-b97d-4809-8580-5dffbab52aca None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:28.071232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 89/177] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:28.076929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c254e8-b97d-4809-8580-5dffbab52aca None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:28.164767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:28.170667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a80b3-c43a-466b-ac89-9e1ea9198d4b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:28.180794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:28.199388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c254e8-b97d-4809-8580-5dffbab52aca None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:28.208002 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41629dc6-19aa-4b16-a9d3-1f8f3c74a635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:28.208625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41629dc6-19aa-4b16-a9d3-1f8f3c74a635 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.208907 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41629dc6-19aa-4b16-a9d3-1f8f3c74a635 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.209274 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41629dc6-19aa-4b16-a9d3-1f8f3c74a635 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.209999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41629dc6-19aa-4b16-a9d3-1f8f3c74a635 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:28.210304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41629dc6-19aa-4b16-a9d3-1f8f3c74a635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:28.211689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41629dc6-19aa-4b16-a9d3-1f8f3c74a635 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:28.219708 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 89/178] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:28.220892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f8240-154a-4563-aa56-ab5c616a740d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.221208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f8240-154a-4563-aa56-ab5c616a740d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.221487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f8240-154a-4563-aa56-ab5c616a740d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.232664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095fa9b-d178-4e0b-a021-65ddef045220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:28.233931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095fa9b-d178-4e0b-a021-65ddef045220 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.234215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095fa9b-d178-4e0b-a021-65ddef045220 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.234489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095fa9b-d178-4e0b-a021-65ddef045220 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.239035 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-257f8240-154a-4563-aa56-ab5c616a740d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:28.240219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-257f8240-154a-4563-aa56-ab5c616a740d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:28.308233 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095fa9b-d178-4e0b-a021-65ddef045220 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:28.309753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095fa9b-d178-4e0b-a021-65ddef045220 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:28.316912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 90/179] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:28.319909 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f8240-154a-4563-aa56-ab5c616a740d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:28.399085 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:28.418869 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 90/180] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:28.420463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550324-616d-4ee0-a145-2d00869f6a36 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.420752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550324-616d-4ee0-a145-2d00869f6a36 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.421092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f550324-616d-4ee0-a145-2d00869f6a36 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.422488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 91/181] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:28.423576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafab948-7a8f-4585-8123-fddc52154591 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.423821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafab948-7a8f-4585-8123-fddc52154591 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.424099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafab948-7a8f-4585-8123-fddc52154591 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.425470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 92/182] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:28.426015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafab948-7a8f-4585-8123-fddc52154591 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:28.428061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:28.436684 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f8240-154a-4563-aa56-ab5c616a740d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:28.470812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6178204-7927-4bf4-a8f5-efd67569924e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:28.471448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6178204-7927-4bf4-a8f5-efd67569924e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.471730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6178204-7927-4bf4-a8f5-efd67569924e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.472004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6178204-7927-4bf4-a8f5-efd67569924e None admin] PATH_INFO: `/v3/users/e1f80e64c9e340ae828c1fab2ceed44c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.473559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6178204-7927-4bf4-a8f5-efd67569924e None admin] RBAC: Authorizing `identity:get_user(user_id=e1f80e64c9e340ae828c1fab2ceed44c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:28.474993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6178204-7927-4bf4-a8f5-efd67569924e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:28.476850 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 91/183] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:28 2022] GET /identity/v3/users/e1f80e64c9e340ae828c1fab2ceed44c => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:28.477454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6178204-7927-4bf4-a8f5-efd67569924e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:28.537125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafab948-7a8f-4585-8123-fddc52154591 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:28.570617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ad68d-0a91-4e9b-bfa7-77723e3b9ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:28.571453 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ad68d-0a91-4e9b-bfa7-77723e3b9ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.571741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ad68d-0a91-4e9b-bfa7-77723e3b9ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.572021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ad68d-0a91-4e9b-bfa7-77723e3b9ad6 None admin] PATH_INFO: `/v3/users/9c3d93f1023e47a294e3b8fe7a6a387e` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.573246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ad68d-0a91-4e9b-bfa7-77723e3b9ad6 None admin] RBAC: Authorizing `identity:get_user(user_id=9c3d93f1023e47a294e3b8fe7a6a387e)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:28.574700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ad68d-0a91-4e9b-bfa7-77723e3b9ad6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:28.576146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6178204-7927-4bf4-a8f5-efd67569924e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:28.576308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 93/184] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:28 2022] GET /identity/v3/users/9c3d93f1023e47a294e3b8fe7a6a387e => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:28.577179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ad68d-0a91-4e9b-bfa7-77723e3b9ad6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:28.611673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2321598-9876-4f76-8b5a-d45058a6ded5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:28.612311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2321598-9876-4f76-8b5a-d45058a6ded5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.612590 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2321598-9876-4f76-8b5a-d45058a6ded5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.612866 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2321598-9876-4f76-8b5a-d45058a6ded5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.623122 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2321598-9876-4f76-8b5a-d45058a6ded5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:28.626301 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f2321598-9876-4f76-8b5a-d45058a6ded5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:28.628861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:28.629747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:28.630546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 92/185] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:52:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:28.631667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6106-f61c-48ed-9549-2fed47aa84e3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.631888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6106-f61c-48ed-9549-2fed47aa84e3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.632158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6106-f61c-48ed-9549-2fed47aa84e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.649660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b2dd6106-f61c-48ed-9549-2fed47aa84e3 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:28.650843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b2dd6106-f61c-48ed-9549-2fed47aa84e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:28.665871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:28.683490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ad68d-0a91-4e9b-bfa7-77723e3b9ad6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:28.716947 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b6a6d-a0e5-4d43-a43e-9edf6ba9bf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:28.717613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b6a6d-a0e5-4d43-a43e-9edf6ba9bf0e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.717894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b6a6d-a0e5-4d43-a43e-9edf6ba9bf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.718167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b6a6d-a0e5-4d43-a43e-9edf6ba9bf0e None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.719338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b6a6d-a0e5-4d43-a43e-9edf6ba9bf0e None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:28.720456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 93/186] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:28.721443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b6a6d-a0e5-4d43-a43e-9edf6ba9bf0e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:28.721811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0bf91-e629-49b0-a539-d0216d71f311 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.722109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0bf91-e629-49b0-a539-d0216d71f311 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.722389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0bf91-e629-49b0-a539-d0216d71f311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.727974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 94/187] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:28 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:28.727974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b6a6d-a0e5-4d43-a43e-9edf6ba9bf0e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:28.748345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-19c0bf91-e629-49b0-a539-d0216d71f311 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:28.749576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-19c0bf91-e629-49b0-a539-d0216d71f311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:28.815544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 94/188] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:28.817864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0bf91-e629-49b0-a539-d0216d71f311 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:28.828534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b6a6d-a0e5-4d43-a43e-9edf6ba9bf0e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:28.847045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:28.864721 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745f13b-7566-4496-807e-44563c146bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:28.865377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745f13b-7566-4496-807e-44563c146bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.865666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745f13b-7566-4496-807e-44563c146bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.865944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745f13b-7566-4496-807e-44563c146bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:28.898140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:28.917022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0bf91-e629-49b0-a539-d0216d71f311 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:28.933198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745f13b-7566-4496-807e-44563c146bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:28.934672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745f13b-7566-4496-807e-44563c146bf2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:28.951529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05486c-bd36-4709-9b9f-6fa7a5d0473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:28.952154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05486c-bd36-4709-9b9f-6fa7a5d0473c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:28.952442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05486c-bd36-4709-9b9f-6fa7a5d0473c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:28.952715 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05486c-bd36-4709-9b9f-6fa7a5d0473c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.019886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05486c-bd36-4709-9b9f-6fa7a5d0473c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.021387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05486c-bd36-4709-9b9f-6fa7a5d0473c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.033237 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 95/189] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:29.034063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745f13b-7566-4496-807e-44563c146bf2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.051466 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:29.103236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:29.122493 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 95/190] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:29.123468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05486c-bd36-4709-9b9f-6fa7a5d0473c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.135072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745f13b-7566-4496-807e-44563c146bf2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.168823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614238c-71b9-473b-963a-70cc77132ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.169467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614238c-71b9-473b-963a-70cc77132ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.169750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614238c-71b9-473b-963a-70cc77132ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.170298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614238c-71b9-473b-963a-70cc77132ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.170745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614238c-71b9-473b-963a-70cc77132ca3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:29.171058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614238c-71b9-473b-963a-70cc77132ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.172415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614238c-71b9-473b-963a-70cc77132ca3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.180314 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 96/191] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:29.180314 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614238c-71b9-473b-963a-70cc77132ca3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.227606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05486c-bd36-4709-9b9f-6fa7a5d0473c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.261447 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:29.261610 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e556793-f677-4d22-a36c-809c59856383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.262346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e556793-f677-4d22-a36c-809c59856383 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.262584 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e556793-f677-4d22-a36c-809c59856383 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.262855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e556793-f677-4d22-a36c-809c59856383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.263644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e556793-f677-4d22-a36c-809c59856383 None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=de970216eec1471ca4c15bf6365e4ef0, user.id=1dff148688b84b08aaa5c8689269444f) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:29.263952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e556793-f677-4d22-a36c-809c59856383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.265407 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e556793-f677-4d22-a36c-809c59856383 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.271590 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 96/192] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/role_assignments?user.id=1dff148688b84b08aaa5c8689269444f&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:29.272366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e556793-f677-4d22-a36c-809c59856383 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.282117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614238c-71b9-473b-963a-70cc77132ca3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.322675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9159a-a19e-4d5d-9386-baf73271f644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.323752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9159a-a19e-4d5d-9386-baf73271f644 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.324237 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9159a-a19e-4d5d-9386-baf73271f644 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.324622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9159a-a19e-4d5d-9386-baf73271f644 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.329325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:29.336915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9159a-a19e-4d5d-9386-baf73271f644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-74c9159a-a19e-4d5d-9386-baf73271f644 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:29.342750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:29.348917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 97/193] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:29.349926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9159a-a19e-4d5d-9386-baf73271f644 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.398072 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e556793-f677-4d22-a36c-809c59856383 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.445544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aeabf9-c8d0-4539-9a02-6daa29b4b13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.446200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeabf9-c8d0-4539-9a02-6daa29b4b13b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.446483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeabf9-c8d0-4539-9a02-6daa29b4b13b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.446762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeabf9-c8d0-4539-9a02-6daa29b4b13b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.447920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aeabf9-c8d0-4539-9a02-6daa29b4b13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.449393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aeabf9-c8d0-4539-9a02-6daa29b4b13b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.451152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 97/194] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:29.451755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aeabf9-c8d0-4539-9a02-6daa29b4b13b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.470453 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9159a-a19e-4d5d-9386-baf73271f644 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.511260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da89246-fc19-43c6-9b9a-de8fbd567d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.511909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da89246-fc19-43c6-9b9a-de8fbd567d7c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.512193 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da89246-fc19-43c6-9b9a-de8fbd567d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.512474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da89246-fc19-43c6-9b9a-de8fbd567d7c None admin] PATH_INFO: `/v3/projects/8c15a17276f4486bb1d6c95328e3f6f0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.513666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da89246-fc19-43c6-9b9a-de8fbd567d7c None admin] RBAC: Authorizing `identity:get_project(project_id=8c15a17276f4486bb1d6c95328e3f6f0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.515128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da89246-fc19-43c6-9b9a-de8fbd567d7c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.517110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 98/195] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/projects/8c15a17276f4486bb1d6c95328e3f6f0 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:29.517618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da89246-fc19-43c6-9b9a-de8fbd567d7c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.530257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:29.566914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aeabf9-c8d0-4539-9a02-6daa29b4b13b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.579478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:29.602242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f19012-bef8-41a8-9541-ee33f9d696b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.602887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f19012-bef8-41a8-9541-ee33f9d696b3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.603168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f19012-bef8-41a8-9541-ee33f9d696b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.603444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f19012-bef8-41a8-9541-ee33f9d696b3 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.604607 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f19012-bef8-41a8-9541-ee33f9d696b3 None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.606121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f19012-bef8-41a8-9541-ee33f9d696b3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.609312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 98/196] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:29.610733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f19012-bef8-41a8-9541-ee33f9d696b3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.617479 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da89246-fc19-43c6-9b9a-de8fbd567d7c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.653063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84352c-ce15-40bd-b939-3581bc23cfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.653645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84352c-ce15-40bd-b939-3581bc23cfdd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.653930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84352c-ce15-40bd-b939-3581bc23cfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.654215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84352c-ce15-40bd-b939-3581bc23cfdd None admin] PATH_INFO: `/v3/users/c023c26377a54758b37f42418e0d9e00` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.667886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84352c-ce15-40bd-b939-3581bc23cfdd None admin] RBAC: Authorizing `identity:get_user(user_id=c023c26377a54758b37f42418e0d9e00)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.669387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84352c-ce15-40bd-b939-3581bc23cfdd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.671016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 99/197] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/users/c023c26377a54758b37f42418e0d9e00 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:29.671759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84352c-ce15-40bd-b939-3581bc23cfdd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.721630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f19012-bef8-41a8-9541-ee33f9d696b3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.744220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:29.772166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674b58b-35e6-4a51-8d33-619d7596b890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.772800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674b58b-35e6-4a51-8d33-619d7596b890 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.773121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674b58b-35e6-4a51-8d33-619d7596b890 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.773403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674b58b-35e6-4a51-8d33-619d7596b890 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.774132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d674b58b-35e6-4a51-8d33-619d7596b890 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:29.774458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d674b58b-35e6-4a51-8d33-619d7596b890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.775855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d674b58b-35e6-4a51-8d33-619d7596b890 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.783901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 99/198] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:29.784684 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674b58b-35e6-4a51-8d33-619d7596b890 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.786374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84352c-ce15-40bd-b939-3581bc23cfdd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.828260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea70402-d04b-4c77-9504-d55d4ee9d684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.829470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea70402-d04b-4c77-9504-d55d4ee9d684 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.830153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:29.830234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea70402-d04b-4c77-9504-d55d4ee9d684 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.830234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea70402-d04b-4c77-9504-d55d4ee9d684 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.830737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea70402-d04b-4c77-9504-d55d4ee9d684 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.832172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea70402-d04b-4c77-9504-d55d4ee9d684 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.885410 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674b58b-35e6-4a51-8d33-619d7596b890 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.918431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33617ad-a55c-448f-bad3-0957e6e9eb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.918966 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33617ad-a55c-448f-bad3-0957e6e9eb5b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.919250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33617ad-a55c-448f-bad3-0957e6e9eb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.919531 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33617ad-a55c-448f-bad3-0957e6e9eb5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.920335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33617ad-a55c-448f-bad3-0957e6e9eb5b None admin] RBAC: Adding query filter params (role.id=2dc5251f14e945e38ad1777a5f235180, scope.project.id=8c15a17276f4486bb1d6c95328e3f6f0, user.id=e1f80e64c9e340ae828c1fab2ceed44c) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:29.920643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33617ad-a55c-448f-bad3-0957e6e9eb5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.922124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33617ad-a55c-448f-bad3-0957e6e9eb5b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.928329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 100/199] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/role_assignments?user.id=e1f80e64c9e340ae828c1fab2ceed44c&scope.project.id=8c15a17276f4486bb1d6c95328e3f6f0&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:29.938759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 100/200] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:52:29 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:29.939015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33617ad-a55c-448f-bad3-0957e6e9eb5b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.939100 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea70402-d04b-4c77-9504-d55d4ee9d684 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.076761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea70402-d04b-4c77-9504-d55d4ee9d684 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.085294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33617ad-a55c-448f-bad3-0957e6e9eb5b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.094877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:30.116334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:30.128955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b00d93-bc0d-446b-a5aa-da6b2247a241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.129615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00d93-bc0d-446b-a5aa-da6b2247a241 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.129898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00d93-bc0d-446b-a5aa-da6b2247a241 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.130190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00d93-bc0d-446b-a5aa-da6b2247a241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.139690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4aebc9-1846-4e43-ac85-f89fc775e726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.140330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aebc9-1846-4e43-ac85-f89fc775e726 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.140614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aebc9-1846-4e43-ac85-f89fc775e726 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.140897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aebc9-1846-4e43-ac85-f89fc775e726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.141838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4aebc9-1846-4e43-ac85-f89fc775e726 None admin] RBAC: Adding query filter params (role.id=45dbb625c3064606a023cf5d620476eb, scope.project.id=de970216eec1471ca4c15bf6365e4ef0, user.id=9c3d93f1023e47a294e3b8fe7a6a387e) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:30.142151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4aebc9-1846-4e43-ac85-f89fc775e726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.148158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4aebc9-1846-4e43-ac85-f89fc775e726 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.150986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 101/201] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/role_assignments?user.id=9c3d93f1023e47a294e3b8fe7a6a387e&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=45dbb625c3064606a023cf5d620476eb => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:30.152946 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4aebc9-1846-4e43-ac85-f89fc775e726 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.239279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b00d93-bc0d-446b-a5aa-da6b2247a241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.240883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b00d93-bc0d-446b-a5aa-da6b2247a241 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.306431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4aebc9-1846-4e43-ac85-f89fc775e726 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.348868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abcb45-ec31-4529-b476-a5cb57195778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.349583 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abcb45-ec31-4529-b476-a5cb57195778 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.349868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abcb45-ec31-4529-b476-a5cb57195778 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.350151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abcb45-ec31-4529-b476-a5cb57195778 None admin] PATH_INFO: `/v3/users/c023c26377a54758b37f42418e0d9e00` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.351395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abcb45-ec31-4529-b476-a5cb57195778 None admin] RBAC: Authorizing `identity:get_user(user_id=c023c26377a54758b37f42418e0d9e00)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.352853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abcb45-ec31-4529-b476-a5cb57195778 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.355034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 102/202] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:30 2022] GET /identity/v3/users/c023c26377a54758b37f42418e0d9e00 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:30.361264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abcb45-ec31-4529-b476-a5cb57195778 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.374724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 101/203] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:30.384470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b00d93-bc0d-446b-a5aa-da6b2247a241 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.417111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:30.425263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:30.503011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abcb45-ec31-4529-b476-a5cb57195778 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.534874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b00d93-bc0d-446b-a5aa-da6b2247a241 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.539176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432de0b-dcc9-4c66-b9e8-f0dcd301dc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.539807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432de0b-dcc9-4c66-b9e8-f0dcd301dc6b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.540090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432de0b-dcc9-4c66-b9e8-f0dcd301dc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.540367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432de0b-dcc9-4c66-b9e8-f0dcd301dc6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.580475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a2625-52c8-4ed9-9440-4f27aa2089d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.581345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2625-52c8-4ed9-9440-4f27aa2089d2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.581657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2625-52c8-4ed9-9440-4f27aa2089d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.581938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2625-52c8-4ed9-9440-4f27aa2089d2 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.583753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a2625-52c8-4ed9-9440-4f27aa2089d2 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.585747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a2625-52c8-4ed9-9440-4f27aa2089d2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.588564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 102/204] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:30 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:30.595314 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a2625-52c8-4ed9-9440-4f27aa2089d2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.618617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432de0b-dcc9-4c66-b9e8-f0dcd301dc6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.620074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432de0b-dcc9-4c66-b9e8-f0dcd301dc6b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.675418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:30.704836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:30.716525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a2625-52c8-4ed9-9440-4f27aa2089d2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.759101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87ba46-e2a6-4b4d-bfcf-84c6857f59ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.759785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87ba46-e2a6-4b4d-bfcf-84c6857f59ec None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.760167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87ba46-e2a6-4b4d-bfcf-84c6857f59ec None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.760546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87ba46-e2a6-4b4d-bfcf-84c6857f59ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.761488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87ba46-e2a6-4b4d-bfcf-84c6857f59ec None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=1e2fa6632a6f4745b4a2406290657577, user.id=c023c26377a54758b37f42418e0d9e00) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:30.761906 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87ba46-e2a6-4b4d-bfcf-84c6857f59ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.762949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 103/205] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:30.763580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87ba46-e2a6-4b4d-bfcf-84c6857f59ec None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.772011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432de0b-dcc9-4c66-b9e8-f0dcd301dc6b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.777849 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 103/206] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:30 2022] GET /identity/v3/role_assignments?user.id=c023c26377a54758b37f42418e0d9e00&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:30.889979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432de0b-dcc9-4c66-b9e8-f0dcd301dc6b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.928564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8152ba3-7e94-48d4-953d-f78ee704babf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.929257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8152ba3-7e94-48d4-953d-f78ee704babf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.929540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8152ba3-7e94-48d4-953d-f78ee704babf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.929823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8152ba3-7e94-48d4-953d-f78ee704babf None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.931722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8152ba3-7e94-48d4-953d-f78ee704babf None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.933263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8152ba3-7e94-48d4-953d-f78ee704babf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.935126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 104/207] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:30 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:30.941452 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87ba46-e2a6-4b4d-bfcf-84c6857f59ec None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:31.050998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87ba46-e2a6-4b4d-bfcf-84c6857f59ec None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:31.088430 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf2424-e536-420f-890a-258fff1d2e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:31.089086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf2424-e536-420f-890a-258fff1d2e50 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:31.089372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf2424-e536-420f-890a-258fff1d2e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:31.094434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf2424-e536-420f-890a-258fff1d2e50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:31.094434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf2424-e536-420f-890a-258fff1d2e50 None admin] RBAC: Adding query filter params (role.id=f4c7e5a278e34f03a86f1f88ed77db0c, scope.project.id=1e2fa6632a6f4745b4a2406290657577, user.id=c023c26377a54758b37f42418e0d9e00) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:31.094434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf2424-e536-420f-890a-258fff1d2e50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:31.094434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf2424-e536-420f-890a-258fff1d2e50 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:31.098255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 104/208] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:30 2022] GET /identity/v3/role_assignments?user.id=c023c26377a54758b37f42418e0d9e00&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=f4c7e5a278e34f03a86f1f88ed77db0c => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:32.263179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6a5f0-4f58-4109-9d93-461624a50f93 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:32.263702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6a5f0-4f58-4109-9d93-461624a50f93 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:32.263702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6a5f0-4f58-4109-9d93-461624a50f93 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:32.265352 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 105/209] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:32.271302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e58d0a-7a04-4c73-93a3-99b8101d4a69 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:32.271727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e58d0a-7a04-4c73-93a3-99b8101d4a69 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:32.271905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e58d0a-7a04-4c73-93a3-99b8101d4a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:32.290248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e5e58d0a-7a04-4c73-93a3-99b8101d4a69 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:32.291441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e5e58d0a-7a04-4c73-93a3-99b8101d4a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:32.305773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:32.364808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 105/210] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:32.388337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6a5f0-4f58-4109-9d93-461624a50f93 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:32.397648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:32.444515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ba325-989b-4ae0-8843-766d9890343a None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:32.444840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ba325-989b-4ae0-8843-766d9890343a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:32.445104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ba325-989b-4ae0-8843-766d9890343a None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:32.446482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 106/211] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:52:32.451379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631e17-2887-466f-8be2-ba2c5d450c18 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:32.451663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631e17-2887-466f-8be2-ba2c5d450c18 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:32.451941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631e17-2887-466f-8be2-ba2c5d450c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:32.470267 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-13631e17-2887-466f-8be2-ba2c5d450c18 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:32.471478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-13631e17-2887-466f-8be2-ba2c5d450c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:32.511113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6a5f0-4f58-4109-9d93-461624a50f93 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:32.544170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 107/212] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:32.555016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811939ea-40b6-414b-9426-f34b8d3b597e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:32.555666 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811939ea-40b6-414b-9426-f34b8d3b597e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:32.555944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811939ea-40b6-414b-9426-f34b8d3b597e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:32.556275 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811939ea-40b6-414b-9426-f34b8d3b597e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:32.567631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811939ea-40b6-414b-9426-f34b8d3b597e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:32.570159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13631e17-2887-466f-8be2-ba2c5d450c18 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-811939ea-40b6-414b-9426-f34b8d3b597e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:32.573033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:32.574103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:32.575120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:32.575859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 106/213] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:32.581677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811939ea-40b6-414b-9426-f34b8d3b597e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:32.660229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:32.676427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:32.682790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13631e17-2887-466f-8be2-ba2c5d450c18 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:32.720355 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c30f8c-f9c3-45be-bdfa-b02fada53b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:32.721412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c30f8c-f9c3-45be-bdfa-b02fada53b84 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:32.721794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c30f8c-f9c3-45be-bdfa-b02fada53b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:32.722166 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c30f8c-f9c3-45be-bdfa-b02fada53b84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:32.722905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811939ea-40b6-414b-9426-f34b8d3b597e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:32.723668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c30f8c-f9c3-45be-bdfa-b02fada53b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:32.726554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c30f8c-f9c3-45be-bdfa-b02fada53b84 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:32.732412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 108/214] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:32.733724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c30f8c-f9c3-45be-bdfa-b02fada53b84 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:32.767447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b47180-c524-4eb7-aea2-af970d26914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:32.768755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b47180-c524-4eb7-aea2-af970d26914d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:32.769476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b47180-c524-4eb7-aea2-af970d26914d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:32.769902 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b47180-c524-4eb7-aea2-af970d26914d None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:32.770804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b47180-c524-4eb7-aea2-af970d26914d None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:32.771254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b47180-c524-4eb7-aea2-af970d26914d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:32.773700 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b47180-c524-4eb7-aea2-af970d26914d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:32.786933 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 107/215] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:32.792725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b47180-c524-4eb7-aea2-af970d26914d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:32.862873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c30f8c-f9c3-45be-bdfa-b02fada53b84 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:32.904298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b47180-c524-4eb7-aea2-af970d26914d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:32.908796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8f0a1-527c-4f75-9946-86858c272143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:32.909520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8f0a1-527c-4f75-9946-86858c272143 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:32.909808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8f0a1-527c-4f75-9946-86858c272143 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:32.910288 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8f0a1-527c-4f75-9946-86858c272143 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:32.911067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8f0a1-527c-4f75-9946-86858c272143 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:32.912571 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8f0a1-527c-4f75-9946-86858c272143 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:32.932834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:32.940380 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeeeaf4-b57c-433e-af47-360d002dc978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:32.941048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeeaf4-b57c-433e-af47-360d002dc978 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:32.941328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeeaf4-b57c-433e-af47-360d002dc978 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:32.941601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeeaf4-b57c-433e-af47-360d002dc978 None admin] PATH_INFO: `/v3/users/665539dbc6764127a13f66f931b74428` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:32.954854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeeeaf4-b57c-433e-af47-360d002dc978 None admin] RBAC: Authorizing `identity:get_user(user_id=665539dbc6764127a13f66f931b74428)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:32.956293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeeeaf4-b57c-433e-af47-360d002dc978 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:32.957988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 108/216] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:32 2022] GET /identity/v3/users/665539dbc6764127a13f66f931b74428 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:32.962264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd39d1-cc32-47b9-8019-ea7e5cba7afb None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:32.962555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd39d1-cc32-47b9-8019-ea7e5cba7afb None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:32.962866 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd39d1-cc32-47b9-8019-ea7e5cba7afb None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:32.964134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 109/217] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:32.964890 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd39d1-cc32-47b9-8019-ea7e5cba7afb None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:33.022472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 109/218] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:52:32 2022] POST /identity/v3/users => generated 297 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:52:33.033022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad47cb8-ef82-4126-b93b-91347404bacf None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:33.033022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad47cb8-ef82-4126-b93b-91347404bacf None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:33.033022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad47cb8-ef82-4126-b93b-91347404bacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:33.040868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:33.060898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-dad47cb8-ef82-4126-b93b-91347404bacf None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:33.063175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-dad47cb8-ef82-4126-b93b-91347404bacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:33.096997 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd39d1-cc32-47b9-8019-ea7e5cba7afb None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:33.148236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35396d48-2ce9-484f-b928-bd8594c9d1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:33.148795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35396d48-2ce9-484f-b928-bd8594c9d1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:33.149115 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35396d48-2ce9-484f-b928-bd8594c9d1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:33.149398 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35396d48-2ce9-484f-b928-bd8594c9d1e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:33.174772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 110/219] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:33.176096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7bc53-0b41-4b4a-8278-574b27c96562 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:33.176386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7bc53-0b41-4b4a-8278-574b27c96562 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:33.176661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7bc53-0b41-4b4a-8278-574b27c96562 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:33.179303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 111/220] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:33.194492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a404de-80bb-4cab-9684-014c8d8a7791 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:33.194821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a404de-80bb-4cab-9684-014c8d8a7791 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:33.195048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a404de-80bb-4cab-9684-014c8d8a7791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:33.226236 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d6a404de-80bb-4cab-9684-014c8d8a7791 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:33.226746 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d6a404de-80bb-4cab-9684-014c8d8a7791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:33.233221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:33.250298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35396d48-2ce9-484f-b928-bd8594c9d1e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:33.251396 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35396d48-2ce9-484f-b928-bd8594c9d1e6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:33.340421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 112/221] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:33.340421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a404de-80bb-4cab-9684-014c8d8a7791 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:33.404273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 110/222] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:33.405316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35396d48-2ce9-484f-b928-bd8594c9d1e6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:33.478221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:33.509324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:33.525799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35396d48-2ce9-484f-b928-bd8594c9d1e6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:33.564557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce76d7-ae23-49db-8d35-55715a8710c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:33.565528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce76d7-ae23-49db-8d35-55715a8710c3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:33.566003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce76d7-ae23-49db-8d35-55715a8710c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:33.566456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce76d7-ae23-49db-8d35-55715a8710c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:33.578423 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a404de-80bb-4cab-9684-014c8d8a7791 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:33.579084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce76d7-ae23-49db-8d35-55715a8710c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a6ce76d7-ae23-49db-8d35-55715a8710c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:33.585131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:33.586044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:33.586849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:33.589278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 111/223] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:52:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:33.590313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce76d7-ae23-49db-8d35-55715a8710c3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:33.626716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4bc2b-6e76-49f7-9ead-b57a765506d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:33.627632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4bc2b-6e76-49f7-9ead-b57a765506d5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:33.627915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4bc2b-6e76-49f7-9ead-b57a765506d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:33.628191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4bc2b-6e76-49f7-9ead-b57a765506d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:33.639405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4bc2b-6e76-49f7-9ead-b57a765506d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-23e4bc2b-6e76-49f7-9ead-b57a765506d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:33.645185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:33.646077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:33.646890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 113/224] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:52:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:33.647548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4bc2b-6e76-49f7-9ead-b57a765506d5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:33.704277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce76d7-ae23-49db-8d35-55715a8710c3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:33.742772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaa3d3-1532-4f6b-8195-7022ba928a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:33.743790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaa3d3-1532-4f6b-8195-7022ba928a25 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:33.744279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaa3d3-1532-4f6b-8195-7022ba928a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:33.744772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaa3d3-1532-4f6b-8195-7022ba928a25 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:33.746219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddaa3d3-1532-4f6b-8195-7022ba928a25 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:33.747889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddaa3d3-1532-4f6b-8195-7022ba928a25 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:33.749832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 112/225] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:33 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:33.751327 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaa3d3-1532-4f6b-8195-7022ba928a25 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:33.760420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4bc2b-6e76-49f7-9ead-b57a765506d5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:33.770758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:33.788631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:33.804317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad5cde-f081-4f43-835f-7d287b4d1358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:33.805512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad5cde-f081-4f43-835f-7d287b4d1358 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:33.806277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad5cde-f081-4f43-835f-7d287b4d1358 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:33.806764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad5cde-f081-4f43-835f-7d287b4d1358 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:33.807734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad5cde-f081-4f43-835f-7d287b4d1358 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:33.808245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad5cde-f081-4f43-835f-7d287b4d1358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:33.810631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad5cde-f081-4f43-835f-7d287b4d1358 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:33.825179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 114/226] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:52:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:33.826644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad5cde-f081-4f43-835f-7d287b4d1358 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:33.863543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaa3d3-1532-4f6b-8195-7022ba928a25 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:33.901337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364163ad-9d0b-47aa-af8b-ffafc459edd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:33.901968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364163ad-9d0b-47aa-af8b-ffafc459edd1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:33.902253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364163ad-9d0b-47aa-af8b-ffafc459edd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:33.902534 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364163ad-9d0b-47aa-af8b-ffafc459edd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:33.903267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364163ad-9d0b-47aa-af8b-ffafc459edd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:33.903568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364163ad-9d0b-47aa-af8b-ffafc459edd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:33.905202 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364163ad-9d0b-47aa-af8b-ffafc459edd1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:33.914779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 113/227] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:33.915655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364163ad-9d0b-47aa-af8b-ffafc459edd1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:33.972883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad5cde-f081-4f43-835f-7d287b4d1358 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:34.004207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:34.021263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9476376-e27f-41bd-9ae2-34eba6e7648e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:34.022376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9476376-e27f-41bd-9ae2-34eba6e7648e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:34.023159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9476376-e27f-41bd-9ae2-34eba6e7648e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:34.023913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9476376-e27f-41bd-9ae2-34eba6e7648e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:34.025123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9476376-e27f-41bd-9ae2-34eba6e7648e None admin] RBAC: Adding query filter params (role.id=2dc5251f14e945e38ad1777a5f235180, scope.project.id=1e2fa6632a6f4745b4a2406290657577, user.id=665539dbc6764127a13f66f931b74428) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:34.025942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9476376-e27f-41bd-9ae2-34eba6e7648e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:34.027848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9476376-e27f-41bd-9ae2-34eba6e7648e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:34.028316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364163ad-9d0b-47aa-af8b-ffafc459edd1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:34.035374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 115/228] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:33 2022] GET /identity/v3/role_assignments?user.id=665539dbc6764127a13f66f931b74428&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:34.036420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9476376-e27f-41bd-9ae2-34eba6e7648e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:34.100154 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:34.104451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692adadf-97c0-4464-af82-f764c6528be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:34.105455 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692adadf-97c0-4464-af82-f764c6528be4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:34.105455 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692adadf-97c0-4464-af82-f764c6528be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:34.105709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692adadf-97c0-4464-af82-f764c6528be4 None admin] PATH_INFO: `/v3/users/c023c26377a54758b37f42418e0d9e00` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:34.107800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692adadf-97c0-4464-af82-f764c6528be4 None admin] RBAC: Authorizing `identity:get_user(user_id=c023c26377a54758b37f42418e0d9e00)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:34.109647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692adadf-97c0-4464-af82-f764c6528be4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:34.111149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 114/229] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:33 2022] GET /identity/v3/users/c023c26377a54758b37f42418e0d9e00 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:34.116862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692adadf-97c0-4464-af82-f764c6528be4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:34.164139 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9476376-e27f-41bd-9ae2-34eba6e7648e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:34.206728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f7432-bc9a-449a-bb49-c030210fa42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:34.207721 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f7432-bc9a-449a-bb49-c030210fa42c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:34.208195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f7432-bc9a-449a-bb49-c030210fa42c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:34.208737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f7432-bc9a-449a-bb49-c030210fa42c None admin] PATH_INFO: `/v3/users/c023c26377a54758b37f42418e0d9e00` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:34.210380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f7432-bc9a-449a-bb49-c030210fa42c None admin] RBAC: Authorizing `identity:get_user(user_id=c023c26377a54758b37f42418e0d9e00)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:34.212583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f7432-bc9a-449a-bb49-c030210fa42c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:34.214950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 116/230] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:34 2022] GET /identity/v3/users/c023c26377a54758b37f42418e0d9e00 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:34.220807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f7432-bc9a-449a-bb49-c030210fa42c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:34.295031 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692adadf-97c0-4464-af82-f764c6528be4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:34.336559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfb5ba-0849-4a35-b2f9-86327c9e8918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:34.337545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfb5ba-0849-4a35-b2f9-86327c9e8918 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:34.338546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfb5ba-0849-4a35-b2f9-86327c9e8918 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:34.339038 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfb5ba-0849-4a35-b2f9-86327c9e8918 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:34.340757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfb5ba-0849-4a35-b2f9-86327c9e8918 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:34.342506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfb5ba-0849-4a35-b2f9-86327c9e8918 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:34.344405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 115/231] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:34 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:34.350128 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfb5ba-0849-4a35-b2f9-86327c9e8918 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:34.381379 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:34.407972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f7432-bc9a-449a-bb49-c030210fa42c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:34.434147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:34.463989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369e1a1-acc8-46fb-aaab-dc1e7a0f061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:34.466370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369e1a1-acc8-46fb-aaab-dc1e7a0f061b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:34.467004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369e1a1-acc8-46fb-aaab-dc1e7a0f061b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:34.470259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369e1a1-acc8-46fb-aaab-dc1e7a0f061b None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:34.472111 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369e1a1-acc8-46fb-aaab-dc1e7a0f061b None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:34.475117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369e1a1-acc8-46fb-aaab-dc1e7a0f061b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:34.482432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 117/232] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:34 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:34.485903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369e1a1-acc8-46fb-aaab-dc1e7a0f061b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:34.585499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfb5ba-0849-4a35-b2f9-86327c9e8918 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:34.655198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369e1a1-acc8-46fb-aaab-dc1e7a0f061b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:34.692293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313f7cd-ff59-40b3-9bc6-38b7254bfc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:34.693708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313f7cd-ff59-40b3-9bc6-38b7254bfc53 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:34.694178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313f7cd-ff59-40b3-9bc6-38b7254bfc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:34.694662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313f7cd-ff59-40b3-9bc6-38b7254bfc53 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577/users/c023c26377a54758b37f42418e0d9e00/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:34.697153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b313f7cd-ff59-40b3-9bc6-38b7254bfc53 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2fa6632a6f4745b4a2406290657577, user_id=c023c26377a54758b37f42418e0d9e00, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:34.698783 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b313f7cd-ff59-40b3-9bc6-38b7254bfc53 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:34.707604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548cd77-272e-410e-8ac9-de3f9d8a8214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:34.708595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548cd77-272e-410e-8ac9-de3f9d8a8214 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:34.709101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548cd77-272e-410e-8ac9-de3f9d8a8214 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:34.709564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548cd77-272e-410e-8ac9-de3f9d8a8214 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577/users/c023c26377a54758b37f42418e0d9e00/roles/f4c7e5a278e34f03a86f1f88ed77db0c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:34.712099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548cd77-272e-410e-8ac9-de3f9d8a8214 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2fa6632a6f4745b4a2406290657577, user_id=c023c26377a54758b37f42418e0d9e00, role_id=f4c7e5a278e34f03a86f1f88ed77db0c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:34.713948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548cd77-272e-410e-8ac9-de3f9d8a8214 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:34.764221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 118/233] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:52:34 2022] PUT /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577/users/c023c26377a54758b37f42418e0d9e00/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:34.797646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 116/234] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:52:34 2022] PUT /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577/users/c023c26377a54758b37f42418e0d9e00/roles/f4c7e5a278e34f03a86f1f88ed77db0c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:52:35.422141 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9c689-6484-4048-ba2e-f0dde2cd47fd None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:35.422616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9c689-6484-4048-ba2e-f0dde2cd47fd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:35.422793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9c689-6484-4048-ba2e-f0dde2cd47fd None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:35.424313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 119/235] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:35.427718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25d93-c5e6-404f-8333-f701194bd893 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:35.428011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25d93-c5e6-404f-8333-f701194bd893 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:35.428294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25d93-c5e6-404f-8333-f701194bd893 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:35.429922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 117/236] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:35.435237 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d3d16-0566-41b9-ac28-7903ef8a2dd4 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:35.435523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d3d16-0566-41b9-ac28-7903ef8a2dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:35.435800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d3d16-0566-41b9-ac28-7903ef8a2dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:35.436330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02601fc-1bf4-44b1-8f77-7c51bc95d226 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:35.436795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02601fc-1bf4-44b1-8f77-7c51bc95d226 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:35.437295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02601fc-1bf4-44b1-8f77-7c51bc95d226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:35.456348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f02601fc-1bf4-44b1-8f77-7c51bc95d226 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:35.457862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f02601fc-1bf4-44b1-8f77-7c51bc95d226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:35.462036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e14d3d16-0566-41b9-ac28-7903ef8a2dd4 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:35.467395 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e14d3d16-0566-41b9-ac28-7903ef8a2dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:35.545550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:35.563058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 118/237] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:35.590629 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02601fc-1bf4-44b1-8f77-7c51bc95d226 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:35.593542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:35.595565 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 120/238] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:35.639930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d3d16-0566-41b9-ac28-7903ef8a2dd4 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:35.727021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02601fc-1bf4-44b1-8f77-7c51bc95d226 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:35.783128 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e343a-b69b-486e-a5e4-58136cf04090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:35.784186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e343a-b69b-486e-a5e4-58136cf04090 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:35.784674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e343a-b69b-486e-a5e4-58136cf04090 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:35.785178 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e343a-b69b-486e-a5e4-58136cf04090 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:35.786715 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e343a-b69b-486e-a5e4-58136cf04090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:35.788369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58e343a-b69b-486e-a5e4-58136cf04090 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:35.797479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 119/239] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:35.801582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e343a-b69b-486e-a5e4-58136cf04090 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:35.849559 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d3d16-0566-41b9-ac28-7903ef8a2dd4 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:35.892410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93be3a-d20d-4f3a-8334-58dcbb32bb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:35.893456 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93be3a-d20d-4f3a-8334-58dcbb32bb6e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:35.895008 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93be3a-d20d-4f3a-8334-58dcbb32bb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:35.895497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93be3a-d20d-4f3a-8334-58dcbb32bb6e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:35.907583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93be3a-d20d-4f3a-8334-58dcbb32bb6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-df93be3a-d20d-4f3a-8334-58dcbb32bb6e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:35.913091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:35.913948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:35.914743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:35.921600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 121/240] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:35.923909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93be3a-d20d-4f3a-8334-58dcbb32bb6e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:35.969369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:36.005717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:36.058612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e343a-b69b-486e-a5e4-58136cf04090 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:36.074240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93be3a-d20d-4f3a-8334-58dcbb32bb6e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:36.101916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d8080-c51f-4363-8405-139841921f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:36.104233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d8080-c51f-4363-8405-139841921f6b None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:36.104754 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d8080-c51f-4363-8405-139841921f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:36.106094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d8080-c51f-4363-8405-139841921f6b None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:36.107003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d8080-c51f-4363-8405-139841921f6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:36.111999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d8080-c51f-4363-8405-139841921f6b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:36.134733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172684a6-2b22-452f-b011-e83e2548f514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:36.135722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172684a6-2b22-452f-b011-e83e2548f514 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:36.136188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172684a6-2b22-452f-b011-e83e2548f514 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:36.136642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172684a6-2b22-452f-b011-e83e2548f514 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:36.137584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172684a6-2b22-452f-b011-e83e2548f514 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:36.138071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172684a6-2b22-452f-b011-e83e2548f514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:36.139633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172684a6-2b22-452f-b011-e83e2548f514 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:36.153109 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 122/241] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:36.154939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172684a6-2b22-452f-b011-e83e2548f514 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:36.267118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:36.273404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172684a6-2b22-452f-b011-e83e2548f514 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:36.281959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 120/242] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:52:35 2022] POST /identity/v3/users => generated 293 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:52:36.319613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ddff33-1d1f-4d72-91e4-243f96e47332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:36.320283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddff33-1d1f-4d72-91e4-243f96e47332 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:36.320567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddff33-1d1f-4d72-91e4-243f96e47332 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:36.320842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddff33-1d1f-4d72-91e4-243f96e47332 None admin] PATH_INFO: `/v3/users/8d701e107d2a448582106759abacc7e5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:36.341898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ddff33-1d1f-4d72-91e4-243f96e47332 None admin] RBAC: Authorizing `identity:get_user(user_id=8d701e107d2a448582106759abacc7e5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:36.343552 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ddff33-1d1f-4d72-91e4-243f96e47332 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:36.345605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 123/243] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:36 2022] GET /identity/v3/users/8d701e107d2a448582106759abacc7e5 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:36.351596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d8080-c51f-4363-8405-139841921f6b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:36.511697 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde524-e9c5-421a-953e-e79cd5b557ba None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:36.512242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde524-e9c5-421a-953e-e79cd5b557ba None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:36.512242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde524-e9c5-421a-953e-e79cd5b557ba None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:36.513628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 124/244] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:52:36.519608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d8080-c51f-4363-8405-139841921f6b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:36.520639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debd193-e29a-4b5d-ba63-e4e1afe58920 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:36.520923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debd193-e29a-4b5d-ba63-e4e1afe58920 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:36.521235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debd193-e29a-4b5d-ba63-e4e1afe58920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:36.546446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-4debd193-e29a-4b5d-ba63-e4e1afe58920 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:36.548197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4debd193-e29a-4b5d-ba63-e4e1afe58920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:36.549245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:36.562124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93863d0f-6101-403a-9620-6475fbdb923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:36.563066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93863d0f-6101-403a-9620-6475fbdb923f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:36.563704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93863d0f-6101-403a-9620-6475fbdb923f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:36.564580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93863d0f-6101-403a-9620-6475fbdb923f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:36.633878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 125/245] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:36.657752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93863d0f-6101-403a-9620-6475fbdb923f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:36.659358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93863d0f-6101-403a-9620-6475fbdb923f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:36.660952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4debd193-e29a-4b5d-ba63-e4e1afe58920 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:36.757817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:36.763923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 121/246] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:36.769707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93863d0f-6101-403a-9620-6475fbdb923f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:36.776820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4debd193-e29a-4b5d-ba63-e4e1afe58920 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:36.798896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:36.817563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f93e4-576b-48fb-b603-e831cba83ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:36.818521 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f93e4-576b-48fb-b603-e831cba83ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:36.818972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f93e4-576b-48fb-b603-e831cba83ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:36.820159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f93e4-576b-48fb-b603-e831cba83ea5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:36.833740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f93e4-576b-48fb-b603-e831cba83ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-722f93e4-576b-48fb-b603-e831cba83ea5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:36.839719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:36.840738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:36.841583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:36.845022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 126/247] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:36.851091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f93e4-576b-48fb-b603-e831cba83ea5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:36.894347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93863d0f-6101-403a-9620-6475fbdb923f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:36.931447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf0e80-37e3-40db-b1ba-73d4d509f42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:36.932283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0e80-37e3-40db-b1ba-73d4d509f42f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:36.932660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0e80-37e3-40db-b1ba-73d4d509f42f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:36.933063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0e80-37e3-40db-b1ba-73d4d509f42f None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:36.934567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf0e80-37e3-40db-b1ba-73d4d509f42f None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:36.936135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf0e80-37e3-40db-b1ba-73d4d509f42f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:36.937981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 122/248] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:36 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:36.943902 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf0e80-37e3-40db-b1ba-73d4d509f42f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:36.981668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f93e4-576b-48fb-b603-e831cba83ea5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:37.018572 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e8bd1-470e-43d4-9204-32ddd3521d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:37.019449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e8bd1-470e-43d4-9204-32ddd3521d70 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:37.019937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e8bd1-470e-43d4-9204-32ddd3521d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:37.020361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e8bd1-470e-43d4-9204-32ddd3521d70 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:37.021274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e8bd1-470e-43d4-9204-32ddd3521d70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:37.021748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e8bd1-470e-43d4-9204-32ddd3521d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:37.023291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e8bd1-470e-43d4-9204-32ddd3521d70 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:37.033074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 127/249] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:37.039041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e8bd1-470e-43d4-9204-32ddd3521d70 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:37.048910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:37.049439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:37.056252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf0e80-37e3-40db-b1ba-73d4d509f42f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:37.093579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171622-0643-48bd-b877-bd94e958e6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:37.094871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171622-0643-48bd-b877-bd94e958e6ae None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:37.095157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171622-0643-48bd-b877-bd94e958e6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:37.095430 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171622-0643-48bd-b877-bd94e958e6ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:37.096290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171622-0643-48bd-b877-bd94e958e6ae None admin] RBAC: Adding query filter params (role.id=45dbb625c3064606a023cf5d620476eb, scope.project.id=1e2fa6632a6f4745b4a2406290657577, user.id=8d701e107d2a448582106759abacc7e5) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:37.096598 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171622-0643-48bd-b877-bd94e958e6ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:37.098180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171622-0643-48bd-b877-bd94e958e6ae None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:37.103988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 123/250] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:36 2022] GET /identity/v3/role_assignments?user.id=8d701e107d2a448582106759abacc7e5&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=45dbb625c3064606a023cf5d620476eb => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:37.151741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e8bd1-470e-43d4-9204-32ddd3521d70 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:37.201291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214fd31-a165-4496-806e-914d28f9aa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:37.202814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214fd31-a165-4496-806e-914d28f9aa31 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:37.204178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214fd31-a165-4496-806e-914d28f9aa31 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:37.204938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214fd31-a165-4496-806e-914d28f9aa31 None admin] PATH_INFO: `/v3/users/665539dbc6764127a13f66f931b74428` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:37.207596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214fd31-a165-4496-806e-914d28f9aa31 None admin] RBAC: Authorizing `identity:get_user(user_id=665539dbc6764127a13f66f931b74428)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:37.210086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214fd31-a165-4496-806e-914d28f9aa31 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:37.215945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 128/251] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:37 2022] GET /identity/v3/users/665539dbc6764127a13f66f931b74428 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:37.219044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171622-0643-48bd-b877-bd94e958e6ae None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:37.368222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1ff07-e31a-4b4c-8db0-1352903d1afa None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:37.368622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1ff07-e31a-4b4c-8db0-1352903d1afa None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:37.368802 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171622-0643-48bd-b877-bd94e958e6ae None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:37.368903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1ff07-e31a-4b4c-8db0-1352903d1afa None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:37.370438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 129/252] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:37.376173 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84107b4-4051-4175-bcbd-ec9e9f510aba None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:37.376461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84107b4-4051-4175-bcbd-ec9e9f510aba None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:37.376741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84107b4-4051-4175-bcbd-ec9e9f510aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:37.408766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a84107b4-4051-4175-bcbd-ec9e9f510aba None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:37.410004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a84107b4-4051-4175-bcbd-ec9e9f510aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:37.412399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd50957-e04f-4e78-adfb-c032dc45ecac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:37.413090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50957-e04f-4e78-adfb-c032dc45ecac None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:37.413372 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50957-e04f-4e78-adfb-c032dc45ecac None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:37.413645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50957-e04f-4e78-adfb-c032dc45ecac None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:37.414833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd50957-e04f-4e78-adfb-c032dc45ecac None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:37.416289 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd50957-e04f-4e78-adfb-c032dc45ecac None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:37.418320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 124/253] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:37 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:37.426185 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa2c1-d16e-4abe-bf6f-d68a5eac1a12 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:37.426470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa2c1-d16e-4abe-bf6f-d68a5eac1a12 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:37.426748 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa2c1-d16e-4abe-bf6f-d68a5eac1a12 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:37.428189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 125/254] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:37.436886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afa2c1-d16e-4abe-bf6f-d68a5eac1a12 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:37.491700 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:37.545203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 130/255] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:37.547137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ebaa4-97e4-4b4f-9c6d-74e1f89195cc None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:37.548303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ebaa4-97e4-4b4f-9c6d-74e1f89195cc None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:37.548842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ebaa4-97e4-4b4f-9c6d-74e1f89195cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:37.565043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:37.592912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-624ebaa4-97e4-4b4f-9c6d-74e1f89195cc None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:37.595941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-624ebaa4-97e4-4b4f-9c6d-74e1f89195cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:37.652926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afa2c1-d16e-4abe-bf6f-d68a5eac1a12 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:37.714972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d271049-ee11-41c1-81b9-2cbf4e9a22ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:37.716590 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d271049-ee11-41c1-81b9-2cbf4e9a22ba None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:37.717425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d271049-ee11-41c1-81b9-2cbf4e9a22ba None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:37.718183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d271049-ee11-41c1-81b9-2cbf4e9a22ba None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577/users/665539dbc6764127a13f66f931b74428/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:37.722746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d271049-ee11-41c1-81b9-2cbf4e9a22ba None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2fa6632a6f4745b4a2406290657577, user_id=665539dbc6764127a13f66f931b74428, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:37.725509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d271049-ee11-41c1-81b9-2cbf4e9a22ba None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:37.742853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 131/256] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:37.743871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ebaa4-97e4-4b4f-9c6d-74e1f89195cc None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:37.808877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 126/257] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:52:37 2022] PUT /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577/users/665539dbc6764127a13f66f931b74428/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:37.810099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d271049-ee11-41c1-81b9-2cbf4e9a22ba None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:37.879753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ebaa4-97e4-4b4f-9c6d-74e1f89195cc None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:37.921248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:37.939786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d271049-ee11-41c1-81b9-2cbf4e9a22ba None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:37.947163 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9395d-a491-43b8-af85-9ff9cda809bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:37.947832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9395d-a491-43b8-af85-9ff9cda809bd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:37.948118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9395d-a491-43b8-af85-9ff9cda809bd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:37.948398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9395d-a491-43b8-af85-9ff9cda809bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:37.953717 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:37.965891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9395d-a491-43b8-af85-9ff9cda809bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-e2d9395d-a491-43b8-af85-9ff9cda809bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:37.971594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:37.972509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:37.973446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 132/258] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:52:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:37.980702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9395d-a491-43b8-af85-9ff9cda809bd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:37.997903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46c65d-a762-43ab-a103-110f222cacb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:37.999081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46c65d-a762-43ab-a103-110f222cacb9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:37.999570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46c65d-a762-43ab-a103-110f222cacb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:38.000065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46c65d-a762-43ab-a103-110f222cacb9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:38.019941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46c65d-a762-43ab-a103-110f222cacb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-bb46c65d-a762-43ab-a103-110f222cacb9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:38.033924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:38.034895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:38.035681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:38.038329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 127/259] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:52:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:38.044806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46c65d-a762-43ab-a103-110f222cacb9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:38.101068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9395d-a491-43b8-af85-9ff9cda809bd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:38.137306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870a7b9-0396-44c9-90ff-0102767d5d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:38.137306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870a7b9-0396-44c9-90ff-0102767d5d4e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:38.137306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870a7b9-0396-44c9-90ff-0102767d5d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:38.137306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870a7b9-0396-44c9-90ff-0102767d5d4e None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:38.137716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870a7b9-0396-44c9-90ff-0102767d5d4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:38.137716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870a7b9-0396-44c9-90ff-0102767d5d4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:38.138803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870a7b9-0396-44c9-90ff-0102767d5d4e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:38.149141 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 133/260] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:38.154961 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870a7b9-0396-44c9-90ff-0102767d5d4e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:38.220099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:38.232719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46c65d-a762-43ab-a103-110f222cacb9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:38.287848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870a7b9-0396-44c9-90ff-0102767d5d4e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:38.295054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2f1f7-54fd-4330-a18d-d78e6dbf49e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:38.296002 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2f1f7-54fd-4330-a18d-d78e6dbf49e5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:38.296488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2f1f7-54fd-4330-a18d-d78e6dbf49e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:38.296981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2f1f7-54fd-4330-a18d-d78e6dbf49e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:38.298491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf2f1f7-54fd-4330-a18d-d78e6dbf49e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:38.299000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf2f1f7-54fd-4330-a18d-d78e6dbf49e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:38.300695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf2f1f7-54fd-4330-a18d-d78e6dbf49e5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:38.310587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 128/261] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:52:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:38.317231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2f1f7-54fd-4330-a18d-d78e6dbf49e5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:38.326016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143bb73-61ba-4421-a34b-ce9f1a03de6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:38.326969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143bb73-61ba-4421-a34b-ce9f1a03de6f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:38.327470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143bb73-61ba-4421-a34b-ce9f1a03de6f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:38.327925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143bb73-61ba-4421-a34b-ce9f1a03de6f None admin] PATH_INFO: `/v3/users/c023c26377a54758b37f42418e0d9e00` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:38.329246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143bb73-61ba-4421-a34b-ce9f1a03de6f None admin] RBAC: Authorizing `identity:get_user(user_id=c023c26377a54758b37f42418e0d9e00)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:38.330869 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143bb73-61ba-4421-a34b-ce9f1a03de6f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:38.331290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:38.332611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 134/262] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:38 2022] GET /identity/v3/users/c023c26377a54758b37f42418e0d9e00 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:38.338311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143bb73-61ba-4421-a34b-ce9f1a03de6f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:38.445908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2f1f7-54fd-4330-a18d-d78e6dbf49e5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:38.468193 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143bb73-61ba-4421-a34b-ce9f1a03de6f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:38.486871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425b875-440b-4779-885c-54421961a0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:38.488059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425b875-440b-4779-885c-54421961a0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:38.488550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425b875-440b-4779-885c-54421961a0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:38.489075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425b875-440b-4779-885c-54421961a0f1 None admin] PATH_INFO: `/v3/users/c023c26377a54758b37f42418e0d9e00` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:38.490466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425b875-440b-4779-885c-54421961a0f1 None admin] RBAC: Authorizing `identity:get_user(user_id=c023c26377a54758b37f42418e0d9e00)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:38.492243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425b875-440b-4779-885c-54421961a0f1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:38.494381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:38.495838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 129/263] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:38 2022] GET /identity/v3/users/c023c26377a54758b37f42418e0d9e00 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:38.502299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425b875-440b-4779-885c-54421961a0f1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:38.507710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e600356-923f-4639-8c39-3b1c8d351f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:38.508406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e600356-923f-4639-8c39-3b1c8d351f2d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:38.526099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e600356-923f-4639-8c39-3b1c8d351f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:38.526099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e600356-923f-4639-8c39-3b1c8d351f2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:38.594055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e600356-923f-4639-8c39-3b1c8d351f2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:38.595261 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:38.597700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e600356-923f-4639-8c39-3b1c8d351f2d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:38.626037 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425b875-440b-4779-885c-54421961a0f1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:38.701977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6978c6-91eb-4fd0-9db9-25796cf9df15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:38.703009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6978c6-91eb-4fd0-9db9-25796cf9df15 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:38.703555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6978c6-91eb-4fd0-9db9-25796cf9df15 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:38.704045 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6978c6-91eb-4fd0-9db9-25796cf9df15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:38.789155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 135/264] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:52:38.795578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e600356-923f-4639-8c39-3b1c8d351f2d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:38.826074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:38.854349 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6978c6-91eb-4fd0-9db9-25796cf9df15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:38.856502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6978c6-91eb-4fd0-9db9-25796cf9df15 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:38.917277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e600356-923f-4639-8c39-3b1c8d351f2d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:38.960493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32be662-d325-4cc0-8f94-9bc904543b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:38.961205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32be662-d325-4cc0-8f94-9bc904543b67 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:38.961493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32be662-d325-4cc0-8f94-9bc904543b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:38.961773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32be662-d325-4cc0-8f94-9bc904543b67 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:38.963430 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32be662-d325-4cc0-8f94-9bc904543b67 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:38.965116 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32be662-d325-4cc0-8f94-9bc904543b67 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:38.967383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 136/265] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:38 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:38.976546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32be662-d325-4cc0-8f94-9bc904543b67 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:39.015277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:39.074075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 130/266] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:39.080444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6978c6-91eb-4fd0-9db9-25796cf9df15 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:39.105183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:39.115456 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32be662-d325-4cc0-8f94-9bc904543b67 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:39.156313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3da04-d207-4f07-8ac8-7449db3d2d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:39.156313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3da04-d207-4f07-8ac8-7449db3d2d30 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:39.156313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3da04-d207-4f07-8ac8-7449db3d2d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:39.157253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3da04-d207-4f07-8ac8-7449db3d2d30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:39.158115 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3da04-d207-4f07-8ac8-7449db3d2d30 None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=1e2fa6632a6f4745b4a2406290657577, user.id=c023c26377a54758b37f42418e0d9e00) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:39.158612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3da04-d207-4f07-8ac8-7449db3d2d30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:39.161236 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3da04-d207-4f07-8ac8-7449db3d2d30 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:39.167804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 137/267] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:38 2022] GET /identity/v3/role_assignments?user.id=c023c26377a54758b37f42418e0d9e00&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:39.262725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6978c6-91eb-4fd0-9db9-25796cf9df15 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:39.307971 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc1049-5e02-4c4e-ad48-1f8fc5107620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:39.308711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc1049-5e02-4c4e-ad48-1f8fc5107620 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:39.309084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc1049-5e02-4c4e-ad48-1f8fc5107620 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:39.309366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc1049-5e02-4c4e-ad48-1f8fc5107620 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:39.310548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc1049-5e02-4c4e-ad48-1f8fc5107620 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:39.311991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc1049-5e02-4c4e-ad48-1f8fc5107620 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:39.314655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 131/268] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:39 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:39.322414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3da04-d207-4f07-8ac8-7449db3d2d30 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:39.457450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3da04-d207-4f07-8ac8-7449db3d2d30 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:39.497693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652395f0-e917-4820-a73b-0122eab55867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:39.498322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652395f0-e917-4820-a73b-0122eab55867 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:39.498603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652395f0-e917-4820-a73b-0122eab55867 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:39.498881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652395f0-e917-4820-a73b-0122eab55867 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:39.499680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652395f0-e917-4820-a73b-0122eab55867 None admin] RBAC: Adding query filter params (role.id=f4c7e5a278e34f03a86f1f88ed77db0c, scope.project.id=1e2fa6632a6f4745b4a2406290657577, user.id=c023c26377a54758b37f42418e0d9e00) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:39.499990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652395f0-e917-4820-a73b-0122eab55867 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:39.502246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652395f0-e917-4820-a73b-0122eab55867 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:39.508751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 138/269] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:39 2022] GET /identity/v3/role_assignments?user.id=c023c26377a54758b37f42418e0d9e00&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=f4c7e5a278e34f03a86f1f88ed77db0c => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:39.545784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e160c-b38a-4a9a-8580-875eb0d5225c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:39.546086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e160c-b38a-4a9a-8580-875eb0d5225c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:39.546346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e160c-b38a-4a9a-8580-875eb0d5225c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:39.548312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 132/270] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:39.555106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de347b8-5617-4f53-8828-758c43e156ab None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:39.555410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de347b8-5617-4f53-8828-758c43e156ab None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:39.555668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de347b8-5617-4f53-8828-758c43e156ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:39.576537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1de347b8-5617-4f53-8828-758c43e156ab None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:39.577760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1de347b8-5617-4f53-8828-758c43e156ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:39.593809 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:39.659434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 139/271] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:39.699202 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e160c-b38a-4a9a-8580-875eb0d5225c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:39.768334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:39.836763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e160c-b38a-4a9a-8580-875eb0d5225c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:39.880087 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a4405-6b48-4392-a81a-0f86bfc0b00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:39.880837 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a4405-6b48-4392-a81a-0f86bfc0b00c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:39.881059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a4405-6b48-4392-a81a-0f86bfc0b00c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:39.881329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a4405-6b48-4392-a81a-0f86bfc0b00c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:39.896674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a4405-6b48-4392-a81a-0f86bfc0b00c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e12a4405-6b48-4392-a81a-0f86bfc0b00c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:39.902711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:39.903506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 133/272] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:39.910417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de347b8-5617-4f53-8828-758c43e156ab None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:40.033148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de347b8-5617-4f53-8828-758c43e156ab None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:40.075966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e905f8-75ff-42b8-9332-472f8a932460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:40.077018 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e905f8-75ff-42b8-9332-472f8a932460 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.077569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e905f8-75ff-42b8-9332-472f8a932460 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.078039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e905f8-75ff-42b8-9332-472f8a932460 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.079089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e905f8-75ff-42b8-9332-472f8a932460 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:40.079596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e905f8-75ff-42b8-9332-472f8a932460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:40.081254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e905f8-75ff-42b8-9332-472f8a932460 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:40.092431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 140/273] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:40.098534 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a4405-6b48-4392-a81a-0f86bfc0b00c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:40.211200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a4405-6b48-4392-a81a-0f86bfc0b00c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:40.214058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:40.246043 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9067f-2aaa-4069-b321-f0e551b7aea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:40.246758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9067f-2aaa-4069-b321-f0e551b7aea8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.247097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9067f-2aaa-4069-b321-f0e551b7aea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.247380 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9067f-2aaa-4069-b321-f0e551b7aea8 None admin] PATH_INFO: `/v3/users/cd41d602c7e248b2b9ef73459565f417` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.253282 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206586c-495e-4c61-853a-02ddcf18843b None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.253575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206586c-495e-4c61-853a-02ddcf18843b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.253854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206586c-495e-4c61-853a-02ddcf18843b None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.255535 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 141/274] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:52:40.266534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0e92-3cff-4e11-bf15-46e51f3c46da None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.266953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0e92-3cff-4e11-bf15-46e51f3c46da None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.267306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0e92-3cff-4e11-bf15-46e51f3c46da None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.267494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9067f-2aaa-4069-b321-f0e551b7aea8 None admin] RBAC: Authorizing `identity:get_user(user_id=cd41d602c7e248b2b9ef73459565f417)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:40.268939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 142/275] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:40.269077 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9067f-2aaa-4069-b321-f0e551b7aea8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:40.270000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d9b39-15dd-44a8-ab02-914260774690 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.270379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d9b39-15dd-44a8-ab02-914260774690 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.270785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d9b39-15dd-44a8-ab02-914260774690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.271136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 134/276] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:40 2022] GET /identity/v3/users/cd41d602c7e248b2b9ef73459565f417 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:40.276232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd5411-085c-4b87-a900-374140678651 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.276668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd5411-085c-4b87-a900-374140678651 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.277154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd5411-085c-4b87-a900-374140678651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.295218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8b2d9b39-15dd-44a8-ab02-914260774690 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:40.296332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8b2d9b39-15dd-44a8-ab02-914260774690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:40.296592 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-56fd5411-085c-4b87-a900-374140678651 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:40.297930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-56fd5411-085c-4b87-a900-374140678651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:40.304123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:40.364079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 135/277] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:40.373273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 143/278] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:40.373273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d9b39-15dd-44a8-ab02-914260774690 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:40.387965 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd5411-085c-4b87-a900-374140678651 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:40.480851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d9b39-15dd-44a8-ab02-914260774690 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:40.484749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:40.490580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd5411-085c-4b87-a900-374140678651 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:40.518122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f5aec-6242-4373-9da3-93e08169ab18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:40.519300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f5aec-6242-4373-9da3-93e08169ab18 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.519300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f5aec-6242-4373-9da3-93e08169ab18 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.519492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f5aec-6242-4373-9da3-93e08169ab18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.520121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f5aec-6242-4373-9da3-93e08169ab18 None admin] RBAC: Adding query filter params (role.id=2dc5251f14e945e38ad1777a5f235180, user.id=cd41d602c7e248b2b9ef73459565f417, scope.system=all) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:40.520428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f5aec-6242-4373-9da3-93e08169ab18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:40.521911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f5aec-6242-4373-9da3-93e08169ab18 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:40.526917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 144/279] 10.222.0.86 () {56 vars in 1308 bytes} [Thu Jul 28 17:52:40 2022] GET /identity/v3/role_assignments?user.id=cd41d602c7e248b2b9ef73459565f417&scope.system=all&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:40.527248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43d345-3868-4fe7-b381-3f21ff0653ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:40.528105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43d345-3868-4fe7-b381-3f21ff0653ea None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.528570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43d345-3868-4fe7-b381-3f21ff0653ea None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.529051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43d345-3868-4fe7-b381-3f21ff0653ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.538033 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f5aec-6242-4373-9da3-93e08169ab18 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:40.542124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43d345-3868-4fe7-b381-3f21ff0653ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-4f43d345-3868-4fe7-b381-3f21ff0653ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:40.547955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:40.548931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:40.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 136/280] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:40.556523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:40.557961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43d345-3868-4fe7-b381-3f21ff0653ea None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:40.672786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f5aec-6242-4373-9da3-93e08169ab18 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:40.709008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43d345-3868-4fe7-b381-3f21ff0653ea None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:40.718198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3387d33-faf6-4c03-bc0c-7b19485fcc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:40.719079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387d33-faf6-4c03-bc0c-7b19485fcc60 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.719467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387d33-faf6-4c03-bc0c-7b19485fcc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.719847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387d33-faf6-4c03-bc0c-7b19485fcc60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.721283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3387d33-faf6-4c03-bc0c-7b19485fcc60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:40.722941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3387d33-faf6-4c03-bc0c-7b19485fcc60 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:40.725064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 145/281] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:40.730141 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3387d33-faf6-4c03-bc0c-7b19485fcc60 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:40.741383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d531f9-c901-4b0a-8498-cb036fe1b535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:40.742351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d531f9-c901-4b0a-8498-cb036fe1b535 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.742840 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d531f9-c901-4b0a-8498-cb036fe1b535 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.743350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d531f9-c901-4b0a-8498-cb036fe1b535 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.744191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d531f9-c901-4b0a-8498-cb036fe1b535 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:40.744605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d531f9-c901-4b0a-8498-cb036fe1b535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:40.746157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d531f9-c901-4b0a-8498-cb036fe1b535 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:40.755983 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 137/282] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:40.761576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d531f9-c901-4b0a-8498-cb036fe1b535 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:40.778177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:40.827210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:40.853053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3387d33-faf6-4c03-bc0c-7b19485fcc60 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:40.868555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d531f9-c901-4b0a-8498-cb036fe1b535 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:40.890995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdec2a-c976-49ab-aa2f-aea4913fd112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:40.891740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdec2a-c976-49ab-aa2f-aea4913fd112 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.892123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdec2a-c976-49ab-aa2f-aea4913fd112 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.892419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdec2a-c976-49ab-aa2f-aea4913fd112 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.893156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdec2a-c976-49ab-aa2f-aea4913fd112 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:40.894736 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdec2a-c976-49ab-aa2f-aea4913fd112 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:40.902650 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaea8c-3047-42b1-849e-ad2ec1b71500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:40.903418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaea8c-3047-42b1-849e-ad2ec1b71500 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.903705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaea8c-3047-42b1-849e-ad2ec1b71500 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.903984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaea8c-3047-42b1-849e-ad2ec1b71500 None admin] PATH_INFO: `/v3/users/8d701e107d2a448582106759abacc7e5` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.905550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acaea8c-3047-42b1-849e-ad2ec1b71500 None admin] RBAC: Authorizing `identity:get_user(user_id=8d701e107d2a448582106759abacc7e5)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:40.907610 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acaea8c-3047-42b1-849e-ad2ec1b71500 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:40.909274 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 138/283] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:40 2022] GET /identity/v3/users/8d701e107d2a448582106759abacc7e5 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:40.910546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7ae05-068c-4e5a-b41b-41ec3a460734 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:40.910838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7ae05-068c-4e5a-b41b-41ec3a460734 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:40.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7ae05-068c-4e5a-b41b-41ec3a460734 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:40.912456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 139/284] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:40.920997 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7ae05-068c-4e5a-b41b-41ec3a460734 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:41.002104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:41.003460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 146/285] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:52:40 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:52:41.004822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc3038-723f-4d51-af10-7bb7ee8f052c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:41.005146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc3038-723f-4d51-af10-7bb7ee8f052c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:41.011585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc3038-723f-4d51-af10-7bb7ee8f052c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:41.027794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e5dc3038-723f-4d51-af10-7bb7ee8f052c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:41.029268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e5dc3038-723f-4d51-af10-7bb7ee8f052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:41.032338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7ae05-068c-4e5a-b41b-41ec3a460734 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:41.086451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6d71e-8859-41b8-899e-7f0602bdc684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:41.088090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6d71e-8859-41b8-899e-7f0602bdc684 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:41.088880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6d71e-8859-41b8-899e-7f0602bdc684 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:41.089400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6d71e-8859-41b8-899e-7f0602bdc684 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:41.091766 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6d71e-8859-41b8-899e-7f0602bdc684 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:41.093546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6d71e-8859-41b8-899e-7f0602bdc684 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:41.097675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 140/286] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:40 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:41.107098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6d71e-8859-41b8-899e-7f0602bdc684 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:41.117900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 147/287] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:41.142638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc3038-723f-4d51-af10-7bb7ee8f052c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:41.222917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:41.239058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6d71e-8859-41b8-899e-7f0602bdc684 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:41.275845 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc3038-723f-4d51-af10-7bb7ee8f052c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:41.287846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd68b8-ee99-4f5c-b6fb-6063363ec83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:41.288586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd68b8-ee99-4f5c-b6fb-6063363ec83c None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:41.288871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd68b8-ee99-4f5c-b6fb-6063363ec83c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:41.289183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd68b8-ee99-4f5c-b6fb-6063363ec83c None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577/users/8d701e107d2a448582106759abacc7e5/roles/45dbb625c3064606a023cf5d620476eb` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:41.291203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd68b8-ee99-4f5c-b6fb-6063363ec83c None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2fa6632a6f4745b4a2406290657577, user_id=8d701e107d2a448582106759abacc7e5, role_id=45dbb625c3064606a023cf5d620476eb)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:41.292666 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd68b8-ee99-4f5c-b6fb-6063363ec83c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:41.318786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7f930-0b9f-445c-bbdf-60a27135ff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:41.319507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7f930-0b9f-445c-bbdf-60a27135ff1d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:41.320023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7f930-0b9f-445c-bbdf-60a27135ff1d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:41.320253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7f930-0b9f-445c-bbdf-60a27135ff1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:41.331646 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7f930-0b9f-445c-bbdf-60a27135ff1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-d9d7f930-0b9f-445c-bbdf-60a27135ff1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:41.336704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:41.337650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:41.338455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 148/288] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:41.345139 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7f930-0b9f-445c-bbdf-60a27135ff1d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:41.376418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 141/289] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:52:41 2022] PUT /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577/users/8d701e107d2a448582106759abacc7e5/roles/45dbb625c3064606a023cf5d620476eb => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:41.472604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7f930-0b9f-445c-bbdf-60a27135ff1d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:41.495751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:41.510959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce10c3f-6808-43fb-9b75-ff9fa49282d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:41.511595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10c3f-6808-43fb-9b75-ff9fa49282d5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:41.513437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10c3f-6808-43fb-9b75-ff9fa49282d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:41.513437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10c3f-6808-43fb-9b75-ff9fa49282d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:41.513437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce10c3f-6808-43fb-9b75-ff9fa49282d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:41.513437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce10c3f-6808-43fb-9b75-ff9fa49282d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:41.514647 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce10c3f-6808-43fb-9b75-ff9fa49282d5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:41.525188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 149/290] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:41.530763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd68b8-ee99-4f5c-b6fb-6063363ec83c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:41.558751 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:41.678219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd68b8-ee99-4f5c-b6fb-6063363ec83c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:41.718507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875b19d-4b81-431b-898f-b8eee83fd8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:41.719269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875b19d-4b81-431b-898f-b8eee83fd8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:41.719553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875b19d-4b81-431b-898f-b8eee83fd8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:41.719829 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875b19d-4b81-431b-898f-b8eee83fd8e8 None admin] PATH_INFO: `/v3/users/665539dbc6764127a13f66f931b74428` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:41.721130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875b19d-4b81-431b-898f-b8eee83fd8e8 None admin] RBAC: Authorizing `identity:get_user(user_id=665539dbc6764127a13f66f931b74428)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:41.722603 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875b19d-4b81-431b-898f-b8eee83fd8e8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:41.724575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 142/291] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:41 2022] GET /identity/v3/users/665539dbc6764127a13f66f931b74428 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:41.730252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce10c3f-6808-43fb-9b75-ff9fa49282d5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:41.860868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce10c3f-6808-43fb-9b75-ff9fa49282d5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:41.906948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a2ee8-a3ae-4fff-863a-0b3893991d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:41.907579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a2ee8-a3ae-4fff-863a-0b3893991d19 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:41.907866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a2ee8-a3ae-4fff-863a-0b3893991d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:41.908146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a2ee8-a3ae-4fff-863a-0b3893991d19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:41.959586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:41.981748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a2ee8-a3ae-4fff-863a-0b3893991d19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:41.983217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a2ee8-a3ae-4fff-863a-0b3893991d19 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:42.144401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 150/292] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:42.150340 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875b19d-4b81-431b-898f-b8eee83fd8e8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:42.232674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:42.262292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875b19d-4b81-431b-898f-b8eee83fd8e8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:42.308477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8463d-0b1d-47bc-b41c-189ccb2ed753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:42.309155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8463d-0b1d-47bc-b41c-189ccb2ed753 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:42.309440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8463d-0b1d-47bc-b41c-189ccb2ed753 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:42.309719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8463d-0b1d-47bc-b41c-189ccb2ed753 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:42.310905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8463d-0b1d-47bc-b41c-189ccb2ed753 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:42.312357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8463d-0b1d-47bc-b41c-189ccb2ed753 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:42.314776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 143/293] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:42 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:42.320785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a2ee8-a3ae-4fff-863a-0b3893991d19 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:42.459946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:42.485770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a2ee8-a3ae-4fff-863a-0b3893991d19 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:42.539926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6df5a4-be14-4d5c-a58f-a2a11a2ce900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:42.540553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6df5a4-be14-4d5c-a58f-a2a11a2ce900 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:42.540834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6df5a4-be14-4d5c-a58f-a2a11a2ce900 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:42.541134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6df5a4-be14-4d5c-a58f-a2a11a2ce900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:42.541933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6df5a4-be14-4d5c-a58f-a2a11a2ce900 None admin] RBAC: Adding query filter params (role.id=2dc5251f14e945e38ad1777a5f235180, scope.project.id=1e2fa6632a6f4745b4a2406290657577, user.id=665539dbc6764127a13f66f931b74428) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:42.542241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6df5a4-be14-4d5c-a58f-a2a11a2ce900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:42.543820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6df5a4-be14-4d5c-a58f-a2a11a2ce900 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:42.550728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 151/294] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:42 2022] GET /identity/v3/role_assignments?user.id=665539dbc6764127a13f66f931b74428&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:42.896782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7970012-178a-41a2-a011-2fe7b4c9091c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:42.897171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7970012-178a-41a2-a011-2fe7b4c9091c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:42.897369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7970012-178a-41a2-a011-2fe7b4c9091c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:42.898960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 144/295] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:42.905000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02447cb-9881-43f5-b022-6a576a197056 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:42.905523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02447cb-9881-43f5-b022-6a576a197056 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:42.905600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02447cb-9881-43f5-b022-6a576a197056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:42.925220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a02447cb-9881-43f5-b022-6a576a197056 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:42.926395 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a02447cb-9881-43f5-b022-6a576a197056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:43.003023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 152/296] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:43.027331 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7970012-178a-41a2-a011-2fe7b4c9091c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:43.195654 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7970012-178a-41a2-a011-2fe7b4c9091c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:43.230933 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8697644-00b9-4c4c-a006-73d7493fb3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:43.231589 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697644-00b9-4c4c-a006-73d7493fb3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:43.231923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697644-00b9-4c4c-a006-73d7493fb3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:43.232204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8697644-00b9-4c4c-a006-73d7493fb3e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:43.243098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8697644-00b9-4c4c-a006-73d7493fb3e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d8697644-00b9-4c4c-a006-73d7493fb3e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:43.248776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:43.249713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:43.252256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 145/297] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:43.257001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02447cb-9881-43f5-b022-6a576a197056 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:43.339843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:43.363443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02447cb-9881-43f5-b022-6a576a197056 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:43.404698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c063de-1142-4350-94d3-b9a4d894f920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:43.405372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c063de-1142-4350-94d3-b9a4d894f920 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:43.405652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c063de-1142-4350-94d3-b9a4d894f920 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:43.405925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c063de-1142-4350-94d3-b9a4d894f920 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:43.406695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c063de-1142-4350-94d3-b9a4d894f920 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:43.407000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c063de-1142-4350-94d3-b9a4d894f920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:43.408385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c063de-1142-4350-94d3-b9a4d894f920 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:43.419664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 153/298] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:43.424486 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8697644-00b9-4c4c-a006-73d7493fb3e8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:43.449406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:43.533986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8697644-00b9-4c4c-a006-73d7493fb3e8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:43.568906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798ca15-b026-4bd9-bd28-4496ad1cdecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:43.569641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798ca15-b026-4bd9-bd28-4496ad1cdecb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:43.569927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798ca15-b026-4bd9-bd28-4496ad1cdecb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:43.570204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798ca15-b026-4bd9-bd28-4496ad1cdecb None admin] PATH_INFO: `/v3/users/cd41d602c7e248b2b9ef73459565f417` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:43.571397 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798ca15-b026-4bd9-bd28-4496ad1cdecb None admin] RBAC: Authorizing `identity:get_user(user_id=cd41d602c7e248b2b9ef73459565f417)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:43.572847 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798ca15-b026-4bd9-bd28-4496ad1cdecb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:43.574796 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 146/299] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:43 2022] GET /identity/v3/users/cd41d602c7e248b2b9ef73459565f417 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:43.582037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c063de-1142-4350-94d3-b9a4d894f920 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:43.633133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f7680-cb01-4a52-a4e6-44482a62bac0 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:43.633449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f7680-cb01-4a52-a4e6-44482a62bac0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:43.633701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f7680-cb01-4a52-a4e6-44482a62bac0 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:43.635187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 147/300] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:43.640175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96411311-9a62-4b89-83cc-da1cf306612d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:43.641232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96411311-9a62-4b89-83cc-da1cf306612d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:43.642051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96411311-9a62-4b89-83cc-da1cf306612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:43.666408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-96411311-9a62-4b89-83cc-da1cf306612d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:43.668399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-96411311-9a62-4b89-83cc-da1cf306612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:43.694999 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c063de-1142-4350-94d3-b9a4d894f920 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:43.731668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e67a8-d985-4704-87b5-0b7b3fbfcf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:43.732673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e67a8-d985-4704-87b5-0b7b3fbfcf2e None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:43.733134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e67a8-d985-4704-87b5-0b7b3fbfcf2e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:43.733556 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e67a8-d985-4704-87b5-0b7b3fbfcf2e None admin] PATH_INFO: `/v3/system/users/cd41d602c7e248b2b9ef73459565f417/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:43.735694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e67a8-d985-4704-87b5-0b7b3fbfcf2e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cd41d602c7e248b2b9ef73459565f417, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:43.737303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e67a8-d985-4704-87b5-0b7b3fbfcf2e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:43.744704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 148/301] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:43.751431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed0f1c-0a8f-4a7c-9dc7-6446ab3d7333 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:43.752021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed0f1c-0a8f-4a7c-9dc7-6446ab3d7333 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:43.752301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed0f1c-0a8f-4a7c-9dc7-6446ab3d7333 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:43.753796 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 149/302] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:43.758758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79f5d1-decb-4c49-a0d0-63501bc0cbc6 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:43.759060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79f5d1-decb-4c49-a0d0-63501bc0cbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:43.759327 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79f5d1-decb-4c49-a0d0-63501bc0cbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:43.767865 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:43.778213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2c79f5d1-decb-4c49-a0d0-63501bc0cbc6 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:43.779408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2c79f5d1-decb-4c49-a0d0-63501bc0cbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:43.830098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 154/303] 10.222.0.86 () {58 vars in 1334 bytes} [Thu Jul 28 17:52:43 2022] PUT /identity/v3/system/users/cd41d602c7e248b2b9ef73459565f417/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:43.833958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e67a8-d985-4704-87b5-0b7b3fbfcf2e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:43.837131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:43.846150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 150/304] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:43.847483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b992372-35b3-4522-a032-9669f75f1bce None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:43.847781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b992372-35b3-4522-a032-9669f75f1bce None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:43.848067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b992372-35b3-4522-a032-9669f75f1bce None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:43.849442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 151/305] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:43.859234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acaf2f-d46a-4bd2-939d-3f33fc6b9e87 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:43.860200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acaf2f-d46a-4bd2-939d-3f33fc6b9e87 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:43.860426 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acaf2f-d46a-4bd2-939d-3f33fc6b9e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:43.881551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b7acaf2f-d46a-4bd2-939d-3f33fc6b9e87 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:43.882800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b7acaf2f-d46a-4bd2-939d-3f33fc6b9e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:43.958198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e67a8-d985-4704-87b5-0b7b3fbfcf2e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:43.965346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 152/306] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:43.966713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acaf2f-d46a-4bd2-939d-3f33fc6b9e87 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:44.012411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8966465-c7cc-4865-a8be-c2927edb4d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:44.013014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8966465-c7cc-4865-a8be-c2927edb4d25 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:44.013307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8966465-c7cc-4865-a8be-c2927edb4d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:44.013600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8966465-c7cc-4865-a8be-c2927edb4d25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:44.029781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8966465-c7cc-4865-a8be-c2927edb4d25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-e8966465-c7cc-4865-a8be-c2927edb4d25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:44.039179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:44.040098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:44.040900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:44.041563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 155/307] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:44.046169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:44.049312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8966465-c7cc-4865-a8be-c2927edb4d25 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:44.099751 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acaf2f-d46a-4bd2-939d-3f33fc6b9e87 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:44.119913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:44.137561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751d111-9bfd-4055-870c-a1d5e61291be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:44.138210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751d111-9bfd-4055-870c-a1d5e61291be None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:44.138489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751d111-9bfd-4055-870c-a1d5e61291be None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:44.138764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751d111-9bfd-4055-870c-a1d5e61291be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:44.139913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751d111-9bfd-4055-870c-a1d5e61291be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:44.141377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751d111-9bfd-4055-870c-a1d5e61291be None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:44.142979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 153/308] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:44.143740 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751d111-9bfd-4055-870c-a1d5e61291be None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:44.165019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8966465-c7cc-4865-a8be-c2927edb4d25 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:44.202298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64f795-446b-46f3-bf40-881fa40a5747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:44.202953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64f795-446b-46f3-bf40-881fa40a5747 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:44.203356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64f795-446b-46f3-bf40-881fa40a5747 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:44.203640 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64f795-446b-46f3-bf40-881fa40a5747 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:44.214283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64f795-446b-46f3-bf40-881fa40a5747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8e64f795-446b-46f3-bf40-881fa40a5747 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:44.220192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:44.221143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:44.221929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 156/309] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:44.222578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64f795-446b-46f3-bf40-881fa40a5747 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:44.249421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751d111-9bfd-4055-870c-a1d5e61291be None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:44.281818 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:44.286793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd39d0-1b99-49f9-9dad-be6dbc536a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:44.287411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd39d0-1b99-49f9-9dad-be6dbc536a99 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:44.287692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd39d0-1b99-49f9-9dad-be6dbc536a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:44.287966 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd39d0-1b99-49f9-9dad-be6dbc536a99 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:44.288692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd39d0-1b99-49f9-9dad-be6dbc536a99 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:44.289016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd39d0-1b99-49f9-9dad-be6dbc536a99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:44.290379 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd39d0-1b99-49f9-9dad-be6dbc536a99 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:44.301143 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 154/310] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:44.301143 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd39d0-1b99-49f9-9dad-be6dbc536a99 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:44.336429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64f795-446b-46f3-bf40-881fa40a5747 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:44.359788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:44.375206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80eae28-c990-4bd8-983a-99060faf5f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:44.376115 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eae28-c990-4bd8-983a-99060faf5f4f None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:44.376535 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eae28-c990-4bd8-983a-99060faf5f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:44.377239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eae28-c990-4bd8-983a-99060faf5f4f None admin] PATH_INFO: `/v3/groups` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:44.378260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80eae28-c990-4bd8-983a-99060faf5f4f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:44.380511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80eae28-c990-4bd8-983a-99060faf5f4f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:44.412580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd39d0-1b99-49f9-9dad-be6dbc536a99 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:44.446196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b626c-b3e0-48f8-9991-2ec94ab49dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:44.446831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b626c-b3e0-48f8-9991-2ec94ab49dfb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:44.447116 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b626c-b3e0-48f8-9991-2ec94ab49dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:44.447393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b626c-b3e0-48f8-9991-2ec94ab49dfb None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:44.448113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b626c-b3e0-48f8-9991-2ec94ab49dfb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:44.448420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b626c-b3e0-48f8-9991-2ec94ab49dfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:44.449839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b626c-b3e0-48f8-9991-2ec94ab49dfb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:44.458600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 157/311] 10.222.0.86 () {60 vars in 1134 bytes} [Thu Jul 28 17:52:44 2022] POST /identity/v3/groups => generated 229 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:44.459445 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80eae28-c990-4bd8-983a-99060faf5f4f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:44.459918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 155/312] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:44.469521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b626c-b3e0-48f8-9991-2ec94ab49dfb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:44.569820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:44.592200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80eae28-c990-4bd8-983a-99060faf5f4f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:44.636155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5a0bd-ef3e-488e-899e-56130ea67bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:44.636995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a0bd-ef3e-488e-899e-56130ea67bfb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:44.637400 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a0bd-ef3e-488e-899e-56130ea67bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:44.637787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a0bd-ef3e-488e-899e-56130ea67bfb None admin] PATH_INFO: `/v3/users/0eb5a2570de44959b79d49728ef8e8a5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:44.639542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b626c-b3e0-48f8-9991-2ec94ab49dfb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:44.655051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5a0bd-ef3e-488e-899e-56130ea67bfb None admin] RBAC: Authorizing `identity:get_user(user_id=0eb5a2570de44959b79d49728ef8e8a5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:44.658086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5a0bd-ef3e-488e-899e-56130ea67bfb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:44.659927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 158/313] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:44 2022] GET /identity/v3/users/0eb5a2570de44959b79d49728ef8e8a5 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:44.666949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5a0bd-ef3e-488e-899e-56130ea67bfb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:44.685575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53babe-3cd2-419a-9e3d-6338eda218d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:44.686232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53babe-3cd2-419a-9e3d-6338eda218d5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:44.686519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53babe-3cd2-419a-9e3d-6338eda218d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:44.686798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53babe-3cd2-419a-9e3d-6338eda218d5 None admin] PATH_INFO: `/v3/users/8d701e107d2a448582106759abacc7e5` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:44.688515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53babe-3cd2-419a-9e3d-6338eda218d5 None admin] RBAC: Authorizing `identity:get_user(user_id=8d701e107d2a448582106759abacc7e5)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:44.690017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53babe-3cd2-419a-9e3d-6338eda218d5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:44.693120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 156/314] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:44 2022] GET /identity/v3/users/8d701e107d2a448582106759abacc7e5 => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:44.697028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53babe-3cd2-419a-9e3d-6338eda218d5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:44.718557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:44.792392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5a0bd-ef3e-488e-899e-56130ea67bfb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:44.799454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53babe-3cd2-419a-9e3d-6338eda218d5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:44.823632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:44.834942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43953d50-1cb3-4f90-8944-f91798aaf189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:44.835599 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43953d50-1cb3-4f90-8944-f91798aaf189 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:44.838958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43953d50-1cb3-4f90-8944-f91798aaf189 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:44.838958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43953d50-1cb3-4f90-8944-f91798aaf189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:44.839140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf27db-6678-42fb-b228-c24bda6cdf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:44.839140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf27db-6678-42fb-b228-c24bda6cdf37 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:44.839140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf27db-6678-42fb-b228-c24bda6cdf37 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:44.840488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf27db-6678-42fb-b228-c24bda6cdf37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:44.841597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf27db-6678-42fb-b228-c24bda6cdf37 None admin] RBAC: Adding query filter params (role.id=45dbb625c3064606a023cf5d620476eb, user.id=0eb5a2570de44959b79d49728ef8e8a5, scope.system=all) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:44.842151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf27db-6678-42fb-b228-c24bda6cdf37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:44.843838 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf27db-6678-42fb-b228-c24bda6cdf37 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:44.853281 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 159/315] 10.222.0.86 () {56 vars in 1308 bytes} [Thu Jul 28 17:52:44 2022] GET /identity/v3/role_assignments?user.id=0eb5a2570de44959b79d49728ef8e8a5&scope.system=all&role.id=45dbb625c3064606a023cf5d620476eb => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:44.916959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43953d50-1cb3-4f90-8944-f91798aaf189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:44.918548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43953d50-1cb3-4f90-8944-f91798aaf189 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:45.032405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 157/316] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:52:45.038253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf27db-6678-42fb-b228-c24bda6cdf37 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:45.143372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf27db-6678-42fb-b228-c24bda6cdf37 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:45.146661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:45.178877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555209f-6e1c-40d1-b185-0a933637c83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:45.179802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555209f-6e1c-40d1-b185-0a933637c83c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:45.180286 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555209f-6e1c-40d1-b185-0a933637c83c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:45.180751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555209f-6e1c-40d1-b185-0a933637c83c None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:45.182196 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555209f-6e1c-40d1-b185-0a933637c83c None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:45.183848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555209f-6e1c-40d1-b185-0a933637c83c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:45.185746 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 160/317] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:45 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:45.191907 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43953d50-1cb3-4f90-8944-f91798aaf189 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:45.217993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:45.296242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43953d50-1cb3-4f90-8944-f91798aaf189 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:45.333390 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de22807-b0ca-40a4-b62d-c3a94bb873c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:45.334321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de22807-b0ca-40a4-b62d-c3a94bb873c8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:45.334785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de22807-b0ca-40a4-b62d-c3a94bb873c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:45.335240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de22807-b0ca-40a4-b62d-c3a94bb873c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:45.336242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de22807-b0ca-40a4-b62d-c3a94bb873c8 None admin] RBAC: Adding query filter params (role.id=45dbb625c3064606a023cf5d620476eb, scope.project.id=1e2fa6632a6f4745b4a2406290657577, user.id=8d701e107d2a448582106759abacc7e5) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:45.336736 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de22807-b0ca-40a4-b62d-c3a94bb873c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:45.338371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de22807-b0ca-40a4-b62d-c3a94bb873c8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:45.349084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 158/318] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:52:45 2022] GET /identity/v3/role_assignments?user.id=8d701e107d2a448582106759abacc7e5&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=45dbb625c3064606a023cf5d620476eb => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:45.918151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea694-5b8d-4a91-ac5a-fbf211fc0457 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:45.918537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea694-5b8d-4a91-ac5a-fbf211fc0457 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:45.918713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea694-5b8d-4a91-ac5a-fbf211fc0457 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:45.920230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 161/319] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:45.925660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f6e4-670d-4a94-96d2-5432f47f8863 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:45.925952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f6e4-670d-4a94-96d2-5432f47f8863 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:45.926230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f6e4-670d-4a94-96d2-5432f47f8863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:45.944524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b1b3f6e4-670d-4a94-96d2-5432f47f8863 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:45.945743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b1b3f6e4-670d-4a94-96d2-5432f47f8863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:46.014376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 159/320] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:46.036790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcea694-5b8d-4a91-ac5a-fbf211fc0457 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:46.175510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcea694-5b8d-4a91-ac5a-fbf211fc0457 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:46.209959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84a296-1eda-489b-ab40-d32c807f95be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:46.210596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84a296-1eda-489b-ab40-d32c807f95be None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.210878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84a296-1eda-489b-ab40-d32c807f95be None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.211151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84a296-1eda-489b-ab40-d32c807f95be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.222381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84a296-1eda-489b-ab40-d32c807f95be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ef84a296-1eda-489b-ab40-d32c807f95be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:46.227431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:46.228313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:46.234106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 162/321] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:46.235425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3f6e4-670d-4a94-96d2-5432f47f8863 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:46.253764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:46.334968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3f6e4-670d-4a94-96d2-5432f47f8863 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:46.370062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526109a1-ff9b-4f0c-be78-55c97d55392c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:46.370690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526109a1-ff9b-4f0c-be78-55c97d55392c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.370968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526109a1-ff9b-4f0c-be78-55c97d55392c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.371243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526109a1-ff9b-4f0c-be78-55c97d55392c None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.371966 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526109a1-ff9b-4f0c-be78-55c97d55392c None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:46.372272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526109a1-ff9b-4f0c-be78-55c97d55392c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:46.373668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526109a1-ff9b-4f0c-be78-55c97d55392c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:46.383044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 160/322] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:46.388495 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84a296-1eda-489b-ab40-d32c807f95be None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:46.428732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:46.507973 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84a296-1eda-489b-ab40-d32c807f95be None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:46.546392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6cfad-3614-4d65-a2f3-ed902e476309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:46.547575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6cfad-3614-4d65-a2f3-ed902e476309 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.547863 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6cfad-3614-4d65-a2f3-ed902e476309 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.548139 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6cfad-3614-4d65-a2f3-ed902e476309 None admin] PATH_INFO: `/v3/users/cd41d602c7e248b2b9ef73459565f417` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.549398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6cfad-3614-4d65-a2f3-ed902e476309 None admin] RBAC: Authorizing `identity:get_user(user_id=cd41d602c7e248b2b9ef73459565f417)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:46.550867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6cfad-3614-4d65-a2f3-ed902e476309 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:46.552709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 163/323] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:46 2022] GET /identity/v3/users/cd41d602c7e248b2b9ef73459565f417 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:46.558208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526109a1-ff9b-4f0c-be78-55c97d55392c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:46.642160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:46.661297 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526109a1-ff9b-4f0c-be78-55c97d55392c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:46.695869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b882a-f89d-4893-a985-dcf2ff3e8ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:46.696412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b882a-f89d-4893-a985-dcf2ff3e8ecc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.696695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b882a-f89d-4893-a985-dcf2ff3e8ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.696990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b882a-f89d-4893-a985-dcf2ff3e8ecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.697798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b882a-f89d-4893-a985-dcf2ff3e8ecc None admin] RBAC: Adding query filter params (role.id=2dc5251f14e945e38ad1777a5f235180, user.id=cd41d602c7e248b2b9ef73459565f417, scope.system=all) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:46.698110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b882a-f89d-4893-a985-dcf2ff3e8ecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:46.699623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b882a-f89d-4893-a985-dcf2ff3e8ecc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:46.701359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576eefe-42f8-457d-9619-599dc3ef6916 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.701796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576eefe-42f8-457d-9619-599dc3ef6916 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.702293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576eefe-42f8-457d-9619-599dc3ef6916 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.703806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 164/324] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:46.708561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa489a2e-51e9-46ef-8ffb-4af5f11c81ba None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.708855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa489a2e-51e9-46ef-8ffb-4af5f11c81ba None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.709167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa489a2e-51e9-46ef-8ffb-4af5f11c81ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.712414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 161/325] 10.222.0.86 () {56 vars in 1308 bytes} [Thu Jul 28 17:52:46 2022] GET /identity/v3/role_assignments?user.id=cd41d602c7e248b2b9ef73459565f417&scope.system=all&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 510 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:46.737982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-aa489a2e-51e9-46ef-8ffb-4af5f11c81ba None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:46.739246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-aa489a2e-51e9-46ef-8ffb-4af5f11c81ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:46.814449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 165/326] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:46.836737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b882a-f89d-4893-a985-dcf2ff3e8ecc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:46.891916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bc64b-be64-4a97-8599-90ab63dc27c2 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.892216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bc64b-be64-4a97-8599-90ab63dc27c2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.892471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bc64b-be64-4a97-8599-90ab63dc27c2 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.893890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 166/327] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:52:46.898915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fdb33-09ba-40e2-b4d6-38eac801c294 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.899206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fdb33-09ba-40e2-b4d6-38eac801c294 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.899482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fdb33-09ba-40e2-b4d6-38eac801c294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.919984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-392fdb33-09ba-40e2-b4d6-38eac801c294 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:46.921211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-392fdb33-09ba-40e2-b4d6-38eac801c294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:46.944385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b882a-f89d-4893-a985-dcf2ff3e8ecc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:46.945278 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:46.978010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09a6ec-5f83-42b2-a592-cae101aa350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:46.978593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09a6ec-5f83-42b2-a592-cae101aa350e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.978882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09a6ec-5f83-42b2-a592-cae101aa350e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.979165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09a6ec-5f83-42b2-a592-cae101aa350e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.980313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09a6ec-5f83-42b2-a592-cae101aa350e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:46.981783 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09a6ec-5f83-42b2-a592-cae101aa350e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:46.983586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 162/328] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:52:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:46.987631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 167/329] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:46.989368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09a6ec-5f83-42b2-a592-cae101aa350e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:46.998033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:47.009439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fdb33-09ba-40e2-b4d6-38eac801c294 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:47.087917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09a6ec-5f83-42b2-a592-cae101aa350e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:47.106615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fdb33-09ba-40e2-b4d6-38eac801c294 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:47.122607 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bc740-3134-4268-b505-5fcfa9fdb839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:47.123227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bc740-3134-4268-b505-5fcfa9fdb839 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:47.123505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bc740-3134-4268-b505-5fcfa9fdb839 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:47.123782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bc740-3134-4268-b505-5fcfa9fdb839 None admin] PATH_INFO: `/v3/groups` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:47.124451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bc740-3134-4268-b505-5fcfa9fdb839 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:47.125931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bc740-3134-4268-b505-5fcfa9fdb839 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:47.140285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3cbb3-338d-44ec-a1ba-f640a43ce303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:47.141178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3cbb3-338d-44ec-a1ba-f640a43ce303 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:47.141460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3cbb3-338d-44ec-a1ba-f640a43ce303 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:47.141732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3cbb3-338d-44ec-a1ba-f640a43ce303 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:47.151843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3cbb3-338d-44ec-a1ba-f640a43ce303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-d8a3cbb3-338d-44ec-a1ba-f640a43ce303 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:47.157386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:47.161831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:47.162726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 168/330] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:47.164423 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3cbb3-338d-44ec-a1ba-f640a43ce303 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:47.186216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:47.229647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 163/331] 10.222.0.86 () {60 vars in 1134 bytes} [Thu Jul 28 17:52:46 2022] POST /identity/v3/groups => generated 226 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:47.266172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3cbb3-338d-44ec-a1ba-f640a43ce303 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:47.304128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e2201-4e83-447c-80b3-4c89dec06c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:47.304766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e2201-4e83-447c-80b3-4c89dec06c9c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:47.305080 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e2201-4e83-447c-80b3-4c89dec06c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:47.305361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e2201-4e83-447c-80b3-4c89dec06c9c None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:47.306104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e2201-4e83-447c-80b3-4c89dec06c9c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:47.306414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e2201-4e83-447c-80b3-4c89dec06c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:47.307807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e2201-4e83-447c-80b3-4c89dec06c9c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:47.317285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 169/332] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:47.322959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bc740-3134-4268-b505-5fcfa9fdb839 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:47.391832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:47.431266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bc740-3134-4268-b505-5fcfa9fdb839 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:47.467690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1898d-97b2-4030-af0a-48193dd61985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:47.468618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1898d-97b2-4030-af0a-48193dd61985 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:47.469523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1898d-97b2-4030-af0a-48193dd61985 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:47.470217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1898d-97b2-4030-af0a-48193dd61985 None admin] PATH_INFO: `/v3/users/0eb5a2570de44959b79d49728ef8e8a5` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:47.471050 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1898d-97b2-4030-af0a-48193dd61985 None admin] RBAC: Authorizing `identity:get_user(user_id=0eb5a2570de44959b79d49728ef8e8a5)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:47.472516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1898d-97b2-4030-af0a-48193dd61985 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:47.474424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 164/333] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:47 2022] GET /identity/v3/users/0eb5a2570de44959b79d49728ef8e8a5 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:47.479999 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e2201-4e83-447c-80b3-4c89dec06c9c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:47.568876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:47.593031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e2201-4e83-447c-80b3-4c89dec06c9c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:47.632763 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea7375-b35e-4a0e-9328-0c77c63c29b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:47.633469 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea7375-b35e-4a0e-9328-0c77c63c29b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:47.633750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea7375-b35e-4a0e-9328-0c77c63c29b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:47.634028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea7375-b35e-4a0e-9328-0c77c63c29b6 None admin] PATH_INFO: `/v3/system/users/0eb5a2570de44959b79d49728ef8e8a5/roles/45dbb625c3064606a023cf5d620476eb` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:47.635511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea7375-b35e-4a0e-9328-0c77c63c29b6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0eb5a2570de44959b79d49728ef8e8a5, role_id=45dbb625c3064606a023cf5d620476eb)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:47.636979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea7375-b35e-4a0e-9328-0c77c63c29b6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:47.707081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 170/334] 10.222.0.86 () {58 vars in 1334 bytes} [Thu Jul 28 17:52:47 2022] PUT /identity/v3/system/users/0eb5a2570de44959b79d49728ef8e8a5/roles/45dbb625c3064606a023cf5d620476eb => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:50.019212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe14d62-2993-42e4-8f68-4c80302374f6 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.020233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe14d62-2993-42e4-8f68-4c80302374f6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.020740 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe14d62-2993-42e4-8f68-4c80302374f6 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.022645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 165/335] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:50.034369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56eada-3a49-4bb1-b86d-c804304707ef None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.034846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56eada-3a49-4bb1-b86d-c804304707ef None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.034920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56eada-3a49-4bb1-b86d-c804304707ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.058642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-fa56eada-3a49-4bb1-b86d-c804304707ef None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:50.063937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fa56eada-3a49-4bb1-b86d-c804304707ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:50.102663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec28c1a-43e9-4d8d-9b5b-6ac52c831675 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.102969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec28c1a-43e9-4d8d-9b5b-6ac52c831675 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.103233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec28c1a-43e9-4d8d-9b5b-6ac52c831675 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.104587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 166/336] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:50.110199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bad064-a634-49fd-8952-435f7904c531 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.110487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bad064-a634-49fd-8952-435f7904c531 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.110767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bad064-a634-49fd-8952-435f7904c531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.141013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b9bad064-a634-49fd-8952-435f7904c531 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:50.142229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b9bad064-a634-49fd-8952-435f7904c531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:50.171764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 171/337] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:50.213848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56eada-3a49-4bb1-b86d-c804304707ef None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:50.273655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 167/338] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:50.274911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51ef34-2882-476f-a667-768e838ee2bf None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.275200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51ef34-2882-476f-a667-768e838ee2bf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.275474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51ef34-2882-476f-a667-768e838ee2bf None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.276756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 168/339] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:50.278172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a01e-8d24-498d-bda9-ac190d30a30f None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.278729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a01e-8d24-498d-bda9-ac190d30a30f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.279206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a01e-8d24-498d-bda9-ac190d30a30f None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.281025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 169/340] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:50.283012 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74533f7-ed98-40fe-a36f-26dabd5054a9 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.283497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74533f7-ed98-40fe-a36f-26dabd5054a9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.284007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74533f7-ed98-40fe-a36f-26dabd5054a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.311397 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e74533f7-ed98-40fe-a36f-26dabd5054a9 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:50.312637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e74533f7-ed98-40fe-a36f-26dabd5054a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:50.331910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:50.372826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56eada-3a49-4bb1-b86d-c804304707ef None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:50.400670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:50.415644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 170/341] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:50.417553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd68ad-9515-4c1e-9865-fb9475c95a24 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.418046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd68ad-9515-4c1e-9865-fb9475c95a24 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.418549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd68ad-9515-4c1e-9865-fb9475c95a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.438243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419a874-607c-4921-9470-f6f132820744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:50.439092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419a874-607c-4921-9470-f6f132820744 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.439199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419a874-607c-4921-9470-f6f132820744 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.439462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419a874-607c-4921-9470-f6f132820744 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.445307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-83cd68ad-9515-4c1e-9865-fb9475c95a24 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:50.446507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-83cd68ad-9515-4c1e-9865-fb9475c95a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:50.461507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419a874-607c-4921-9470-f6f132820744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-7419a874-607c-4921-9470-f6f132820744 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:50.467079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:50.467956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:50.468756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 172/342] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:50.469454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419a874-607c-4921-9470-f6f132820744 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:50.523592 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 171/343] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:50.524513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd68ad-9515-4c1e-9865-fb9475c95a24 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:50.600865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419a874-607c-4921-9470-f6f132820744 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:50.636334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd68ad-9515-4c1e-9865-fb9475c95a24 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:50.642467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9dc96-b342-4697-a9b2-7aae104a52e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:50.643126 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9dc96-b342-4697-a9b2-7aae104a52e0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.643408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9dc96-b342-4697-a9b2-7aae104a52e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.643689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9dc96-b342-4697-a9b2-7aae104a52e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.647076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:50.662133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9dc96-b342-4697-a9b2-7aae104a52e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:50.666486 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-63e9dc96-b342-4697-a9b2-7aae104a52e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:50.667631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:50.668749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 173/344] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:50.669759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9dc96-b342-4697-a9b2-7aae104a52e0 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:50.694403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77769a-cc36-4194-8cb4-57e8de8c2e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:50.695247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77769a-cc36-4194-8cb4-57e8de8c2e4a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.695586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77769a-cc36-4194-8cb4-57e8de8c2e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.700079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77769a-cc36-4194-8cb4-57e8de8c2e4a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.711778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77769a-cc36-4194-8cb4-57e8de8c2e4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-3e77769a-cc36-4194-8cb4-57e8de8c2e4a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:50.717084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:50.718204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:50.719844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:50.721547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 172/345] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:52:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:50.726893 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77769a-cc36-4194-8cb4-57e8de8c2e4a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:50.824041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9dc96-b342-4697-a9b2-7aae104a52e0 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:50.851921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77769a-cc36-4194-8cb4-57e8de8c2e4a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:50.870120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3024a-b6b9-4867-953c-296224eb8e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:50.871083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3024a-b6b9-4867-953c-296224eb8e14 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.871517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3024a-b6b9-4867-953c-296224eb8e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.872192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3024a-b6b9-4867-953c-296224eb8e14 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.873340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3024a-b6b9-4867-953c-296224eb8e14 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:50.873660 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3024a-b6b9-4867-953c-296224eb8e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:50.879163 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3024a-b6b9-4867-953c-296224eb8e14 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:50.892030 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 174/346] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:50.893171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3024a-b6b9-4867-953c-296224eb8e14 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:50.902200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ce2c1-8254-4de7-aaa3-0b56a8ac9090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:50.903097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ce2c1-8254-4de7-aaa3-0b56a8ac9090 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.903381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ce2c1-8254-4de7-aaa3-0b56a8ac9090 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.903662 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ce2c1-8254-4de7-aaa3-0b56a8ac9090 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.917379 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ce2c1-8254-4de7-aaa3-0b56a8ac9090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-769ce2c1-8254-4de7-aaa3-0b56a8ac9090 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:50.923891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:50.924789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:50.925628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 173/347] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:50.926295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ce2c1-8254-4de7-aaa3-0b56a8ac9090 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:50.977635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:50.993810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:51.031773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3024a-b6b9-4867-953c-296224eb8e14 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:51.064875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ce2c1-8254-4de7-aaa3-0b56a8ac9090 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:51.071415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd8069-2897-4e6d-b7a7-a76216a2ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:51.072043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd8069-2897-4e6d-b7a7-a76216a2ef3f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:51.072440 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd8069-2897-4e6d-b7a7-a76216a2ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:51.072720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd8069-2897-4e6d-b7a7-a76216a2ef3f None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:51.073484 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd8069-2897-4e6d-b7a7-a76216a2ef3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:51.073790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd8069-2897-4e6d-b7a7-a76216a2ef3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:51.075341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd8069-2897-4e6d-b7a7-a76216a2ef3f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:51.085505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 175/348] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:51.086439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd8069-2897-4e6d-b7a7-a76216a2ef3f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:51.103576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0af4e8-6b37-4926-ae38-b0b094a1b8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:51.104236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af4e8-6b37-4926-ae38-b0b094a1b8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:51.104519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af4e8-6b37-4926-ae38-b0b094a1b8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:51.104793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af4e8-6b37-4926-ae38-b0b094a1b8b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:51.105554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0af4e8-6b37-4926-ae38-b0b094a1b8b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:51.105861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0af4e8-6b37-4926-ae38-b0b094a1b8b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:51.107255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0af4e8-6b37-4926-ae38-b0b094a1b8b2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:51.115812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 174/349] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:52:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:51.118608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0af4e8-6b37-4926-ae38-b0b094a1b8b2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:51.217657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd8069-2897-4e6d-b7a7-a76216a2ef3f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:51.241781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:51.244067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0af4e8-6b37-4926-ae38-b0b094a1b8b2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:51.254197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:51.261646 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb722ad-316d-4bd3-9aef-7daab59eec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:51.262696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb722ad-316d-4bd3-9aef-7daab59eec5d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:51.262985 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb722ad-316d-4bd3-9aef-7daab59eec5d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:51.263689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb722ad-316d-4bd3-9aef-7daab59eec5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:51.293616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498196ad-d8ed-4e87-9570-24c14a85819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:51.295402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498196ad-d8ed-4e87-9570-24c14a85819a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:51.295876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498196ad-d8ed-4e87-9570-24c14a85819a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:51.296110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498196ad-d8ed-4e87-9570-24c14a85819a None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:51.298199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498196ad-d8ed-4e87-9570-24c14a85819a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:51.298199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498196ad-d8ed-4e87-9570-24c14a85819a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:51.299089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498196ad-d8ed-4e87-9570-24c14a85819a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:51.314015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 175/350] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:51.315609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a3bab-eb53-43ca-b00d-3513611b11f2 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:51.316564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a3bab-eb53-43ca-b00d-3513611b11f2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:51.317113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a3bab-eb53-43ca-b00d-3513611b11f2 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:51.319036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 176/351] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:51.320310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a3bab-eb53-43ca-b00d-3513611b11f2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:51.351021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb722ad-316d-4bd3-9aef-7daab59eec5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:51.355421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb722ad-316d-4bd3-9aef-7daab59eec5d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:51.458412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a3bab-eb53-43ca-b00d-3513611b11f2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:51.494566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 176/352] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:51.495265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb722ad-316d-4bd3-9aef-7daab59eec5d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:51.513813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a6e17-31bc-41c5-b596-69725e8dab41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:51.514681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6e17-31bc-41c5-b596-69725e8dab41 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:51.514969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6e17-31bc-41c5-b596-69725e8dab41 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:51.515243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6e17-31bc-41c5-b596-69725e8dab41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:51.522240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:51.562176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:51.605886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a6e17-31bc-41c5-b596-69725e8dab41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:51.607351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a6e17-31bc-41c5-b596-69725e8dab41 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:51.636446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb722ad-316d-4bd3-9aef-7daab59eec5d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:51.678856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30539b-c66a-4188-82e0-10c07c01920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:51.679493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30539b-c66a-4188-82e0-10c07c01920a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:51.679818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30539b-c66a-4188-82e0-10c07c01920a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:51.680057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30539b-c66a-4188-82e0-10c07c01920a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:51.735415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 177/353] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:51.736360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a6e17-31bc-41c5-b596-69725e8dab41 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:51.763384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30539b-c66a-4188-82e0-10c07c01920a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:51.764856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30539b-c66a-4188-82e0-10c07c01920a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:51.790992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:51.821840 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:51.860344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a6e17-31bc-41c5-b596-69725e8dab41 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:51.887989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 177/354] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:51.889416 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18d298-c91b-4b78-a6f7-2b0e6ccd4785 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:51.889707 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18d298-c91b-4b78-a6f7-2b0e6ccd4785 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:51.889984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18d298-c91b-4b78-a6f7-2b0e6ccd4785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:51.907344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdc66e-7ef3-4b4d-9f7c-90499958aae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:51.908089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdc66e-7ef3-4b4d-9f7c-90499958aae5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:51.908374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdc66e-7ef3-4b4d-9f7c-90499958aae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:51.908472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6b18d298-c91b-4b78-a6f7-2b0e6ccd4785 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:51.908646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdc66e-7ef3-4b4d-9f7c-90499958aae5 None admin] PATH_INFO: `/v3/users/0eb5a2570de44959b79d49728ef8e8a5` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:51.909669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6b18d298-c91b-4b78-a6f7-2b0e6ccd4785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:51.909908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cdc66e-7ef3-4b4d-9f7c-90499958aae5 None admin] RBAC: Authorizing `identity:get_user(user_id=0eb5a2570de44959b79d49728ef8e8a5)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:51.911339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cdc66e-7ef3-4b4d-9f7c-90499958aae5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:51.913029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 178/355] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:52:51 2022] GET /identity/v3/users/0eb5a2570de44959b79d49728ef8e8a5 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:51.914718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdc66e-7ef3-4b4d-9f7c-90499958aae5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:51.993023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 178/356] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:51.994483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25556121-922f-4d14-9cde-a1cd84bf2f6a None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:51.994716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25556121-922f-4d14-9cde-a1cd84bf2f6a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:51.994993 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25556121-922f-4d14-9cde-a1cd84bf2f6a None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:51.996262 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 179/357] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:51.997075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25556121-922f-4d14-9cde-a1cd84bf2f6a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:52.036995 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdc66e-7ef3-4b4d-9f7c-90499958aae5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:52.066785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:52.081633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54905c7-6a69-44c3-857b-48c90ea51574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:52.082367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54905c7-6a69-44c3-857b-48c90ea51574 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.088616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54905c7-6a69-44c3-857b-48c90ea51574 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.088616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54905c7-6a69-44c3-857b-48c90ea51574 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.088616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54905c7-6a69-44c3-857b-48c90ea51574 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:52.088616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54905c7-6a69-44c3-857b-48c90ea51574 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:52.088616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 179/358] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:51 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:52.088616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54905c7-6a69-44c3-857b-48c90ea51574 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:52.092595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:52.129518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25556121-922f-4d14-9cde-a1cd84bf2f6a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:52.175096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a592c-13b6-4e28-be00-8c50b0884735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:52.175723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a592c-13b6-4e28-be00-8c50b0884735 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.176746 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a592c-13b6-4e28-be00-8c50b0884735 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.176746 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a592c-13b6-4e28-be00-8c50b0884735 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.177973 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a592c-13b6-4e28-be00-8c50b0884735 None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:52.180544 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a592c-13b6-4e28-be00-8c50b0884735 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:52.182132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 180/359] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:51 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:52.182729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a592c-13b6-4e28-be00-8c50b0884735 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:52.209429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54905c7-6a69-44c3-857b-48c90ea51574 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:52.247460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadad31-9894-4b81-b1e7-e4c9160187a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:52.248107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadad31-9894-4b81-b1e7-e4c9160187a4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.248461 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadad31-9894-4b81-b1e7-e4c9160187a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.248737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadad31-9894-4b81-b1e7-e4c9160187a4 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.249975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadad31-9894-4b81-b1e7-e4c9160187a4 None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:52.251413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadad31-9894-4b81-b1e7-e4c9160187a4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:52.255030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 180/360] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:52 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:52.255030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295da32f-0152-418b-a1b4-3a4d93d9ce11 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.255030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295da32f-0152-418b-a1b4-3a4d93d9ce11 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.255292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295da32f-0152-418b-a1b4-3a4d93d9ce11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.275273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-295da32f-0152-418b-a1b4-3a4d93d9ce11 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:52.276471 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-295da32f-0152-418b-a1b4-3a4d93d9ce11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:52.308012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a592c-13b6-4e28-be00-8c50b0884735 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:52.328939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:52.350078 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac4f65-e257-4d8e-8a74-5c3baf9a7cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:52.350883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac4f65-e257-4d8e-8a74-5c3baf9a7cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.351173 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac4f65-e257-4d8e-8a74-5c3baf9a7cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.351452 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac4f65-e257-4d8e-8a74-5c3baf9a7cd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.352261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac4f65-e257-4d8e-8a74-5c3baf9a7cd0 None admin] RBAC: Adding query filter params (role.id=45dbb625c3064606a023cf5d620476eb, user.id=0eb5a2570de44959b79d49728ef8e8a5, scope.system=all) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:52.352575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac4f65-e257-4d8e-8a74-5c3baf9a7cd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:52.354057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac4f65-e257-4d8e-8a74-5c3baf9a7cd0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:52.359014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 181/361] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:52.359014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295da32f-0152-418b-a1b4-3a4d93d9ce11 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:52.360304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:52.369436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 181/362] 10.222.0.86 () {56 vars in 1308 bytes} [Thu Jul 28 17:52:52 2022] GET /identity/v3/role_assignments?user.id=0eb5a2570de44959b79d49728ef8e8a5&scope.system=all&role.id=45dbb625c3064606a023cf5d620476eb => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:52.370292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac4f65-e257-4d8e-8a74-5c3baf9a7cd0 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:52.501017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295da32f-0152-418b-a1b4-3a4d93d9ce11 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:52.507551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac4f65-e257-4d8e-8a74-5c3baf9a7cd0 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:52.554911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fad95-8faf-4eae-a9e1-1a0e587aed16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:52.555919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fad95-8faf-4eae-a9e1-1a0e587aed16 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.556422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fad95-8faf-4eae-a9e1-1a0e587aed16 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.556703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fad95-8faf-4eae-a9e1-1a0e587aed16 None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.566425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fad95-8faf-4eae-a9e1-1a0e587aed16 None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:52.567877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fad95-8faf-4eae-a9e1-1a0e587aed16 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:52.569972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 182/363] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:52:52 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:52.570815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fad95-8faf-4eae-a9e1-1a0e587aed16 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:52.578765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe35fc2-b9e8-43d5-bf0f-2f771762e6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:52.578765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe35fc2-b9e8-43d5-bf0f-2f771762e6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.578765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe35fc2-b9e8-43d5-bf0f-2f771762e6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.578765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe35fc2-b9e8-43d5-bf0f-2f771762e6c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.594619 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe35fc2-b9e8-43d5-bf0f-2f771762e6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-9fe35fc2-b9e8-43d5-bf0f-2f771762e6c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:52.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:52.600549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:52.601427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 182/364] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:52:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:52.602182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe35fc2-b9e8-43d5-bf0f-2f771762e6c2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:52.636054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:52.674934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:52.702862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fad95-8faf-4eae-a9e1-1a0e587aed16 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:52.718023 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe35fc2-b9e8-43d5-bf0f-2f771762e6c2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:52.744177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950eba8-e83c-4184-aebb-a18e2266dded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:52.745042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950eba8-e83c-4184-aebb-a18e2266dded None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.745308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950eba8-e83c-4184-aebb-a18e2266dded None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.745590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950eba8-e83c-4184-aebb-a18e2266dded None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.747144 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950eba8-e83c-4184-aebb-a18e2266dded None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:52.748771 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950eba8-e83c-4184-aebb-a18e2266dded None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:52.750796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 183/365] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:52:52 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:52.751718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950eba8-e83c-4184-aebb-a18e2266dded None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:52.758310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c46e5-2539-4d74-9fcf-4be7ff16e410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:52.758959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c46e5-2539-4d74-9fcf-4be7ff16e410 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.759237 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c46e5-2539-4d74-9fcf-4be7ff16e410 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.759509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c46e5-2539-4d74-9fcf-4be7ff16e410 None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.760679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c46e5-2539-4d74-9fcf-4be7ff16e410 None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:52.762543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c46e5-2539-4d74-9fcf-4be7ff16e410 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:52.764437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 183/366] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:52:52 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:52.765006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c46e5-2539-4d74-9fcf-4be7ff16e410 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:52.863979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950eba8-e83c-4184-aebb-a18e2266dded None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:52.873873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:52.878766 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c46e5-2539-4d74-9fcf-4be7ff16e410 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:52.901968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b93130-278d-4830-9ee4-ee7d57040ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:52.902628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b93130-278d-4830-9ee4-ee7d57040ace None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.902911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b93130-278d-4830-9ee4-ee7d57040ace None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.903186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b93130-278d-4830-9ee4-ee7d57040ace None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.914544 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b93130-278d-4830-9ee4-ee7d57040ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b2b93130-278d-4830-9ee4-ee7d57040ace None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:52.920098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:52.921114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:52.922560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 184/367] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:52:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:52.923576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b93130-278d-4830-9ee4-ee7d57040ace None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:52.924290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:52.934515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b17a0-2a25-4167-a454-16429f2f9201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:52.935064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b17a0-2a25-4167-a454-16429f2f9201 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.935343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b17a0-2a25-4167-a454-16429f2f9201 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.935617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b17a0-2a25-4167-a454-16429f2f9201 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.936506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b17a0-2a25-4167-a454-16429f2f9201 None admin] RBAC: Adding query filter params (group.id=11ba0ae8b86745989af94117b91abdd3, role.id=2dc5251f14e945e38ad1777a5f235180, scope.project.id=1e2fa6632a6f4745b4a2406290657577) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:52.937064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b17a0-2a25-4167-a454-16429f2f9201 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:52.938773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b17a0-2a25-4167-a454-16429f2f9201 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:52.944519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 184/368] 10.222.0.86 () {56 vars in 1376 bytes} [Thu Jul 28 17:52:52 2022] GET /identity/v3/role_assignments?group.id=11ba0ae8b86745989af94117b91abdd3&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:52.945345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b17a0-2a25-4167-a454-16429f2f9201 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:53.042481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b93130-278d-4830-9ee4-ee7d57040ace None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:53.060664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b17a0-2a25-4167-a454-16429f2f9201 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:53.095391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a512c4a-7916-424f-987c-10c9165d55e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:53.096830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512c4a-7916-424f-987c-10c9165d55e1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:53.096830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512c4a-7916-424f-987c-10c9165d55e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:53.096830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512c4a-7916-424f-987c-10c9165d55e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:53.097363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a512c4a-7916-424f-987c-10c9165d55e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:53.097663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a512c4a-7916-424f-987c-10c9165d55e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:53.099334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a512c4a-7916-424f-987c-10c9165d55e1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:53.099437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5400f-7f2b-4d5c-be70-8fa64ad849c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:53.100003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5400f-7f2b-4d5c-be70-8fa64ad849c6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:53.100244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5400f-7f2b-4d5c-be70-8fa64ad849c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:53.100518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5400f-7f2b-4d5c-be70-8fa64ad849c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:53.102542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5400f-7f2b-4d5c-be70-8fa64ad849c6 None admin] RBAC: Adding query filter params (group.id=11ba0ae8b86745989af94117b91abdd3, role.id=2dc5251f14e945e38ad1777a5f235180, scope.project.id=de970216eec1471ca4c15bf6365e4ef0) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:53.102542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5400f-7f2b-4d5c-be70-8fa64ad849c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:53.103140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5400f-7f2b-4d5c-be70-8fa64ad849c6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:53.108484 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 185/369] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:52:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:53.109415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a512c4a-7916-424f-987c-10c9165d55e1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:53.110439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 185/370] 10.222.0.86 () {56 vars in 1376 bytes} [Thu Jul 28 17:52:52 2022] GET /identity/v3/role_assignments?group.id=11ba0ae8b86745989af94117b91abdd3&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:53.111288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5400f-7f2b-4d5c-be70-8fa64ad849c6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:53.147501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:53.189897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:53.231336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a512c4a-7916-424f-987c-10c9165d55e1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:53.243032 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5400f-7f2b-4d5c-be70-8fa64ad849c6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:53.275671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb583e-6c72-4254-bff2-b778c62aaaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:53.276822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb583e-6c72-4254-bff2-b778c62aaaaa None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:53.277370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb583e-6c72-4254-bff2-b778c62aaaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:53.277653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb583e-6c72-4254-bff2-b778c62aaaaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:53.278488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb583e-6c72-4254-bff2-b778c62aaaaa None admin] RBAC: Adding query filter params (group.id=11ba0ae8b86745989af94117b91abdd3, role.id=f4c7e5a278e34f03a86f1f88ed77db0c, scope.project.id=de970216eec1471ca4c15bf6365e4ef0) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:53.278805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb583e-6c72-4254-bff2-b778c62aaaaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:53.281895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb583e-6c72-4254-bff2-b778c62aaaaa None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:53.287154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad41f8-b297-46f9-ac66-af5a619af891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:53.288134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad41f8-b297-46f9-ac66-af5a619af891 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:53.288635 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad41f8-b297-46f9-ac66-af5a619af891 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:53.289165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad41f8-b297-46f9-ac66-af5a619af891 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:53.290117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad41f8-b297-46f9-ac66-af5a619af891 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:53.290648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad41f8-b297-46f9-ac66-af5a619af891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:53.292310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad41f8-b297-46f9-ac66-af5a619af891 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:53.295028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 186/371] 10.222.0.86 () {56 vars in 1376 bytes} [Thu Jul 28 17:52:53 2022] GET /identity/v3/role_assignments?group.id=11ba0ae8b86745989af94117b91abdd3&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=f4c7e5a278e34f03a86f1f88ed77db0c => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:53.296453 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb583e-6c72-4254-bff2-b778c62aaaaa None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:53.304257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 186/372] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:53.311017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad41f8-b297-46f9-ac66-af5a619af891 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:53.458538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:53.478500 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb583e-6c72-4254-bff2-b778c62aaaaa None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:53.485746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad41f8-b297-46f9-ac66-af5a619af891 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:53.512018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:53.529464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591446a-cd12-452f-b21a-25e07dfba945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:53.530320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591446a-cd12-452f-b21a-25e07dfba945 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:53.531687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591446a-cd12-452f-b21a-25e07dfba945 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:53.532111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591446a-cd12-452f-b21a-25e07dfba945 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:53.553698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077c87e-378b-497e-8147-0c909f3c1d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:53.554344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077c87e-378b-497e-8147-0c909f3c1d8f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:53.554632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077c87e-378b-497e-8147-0c909f3c1d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:53.554908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077c87e-378b-497e-8147-0c909f3c1d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:53.641193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591446a-cd12-452f-b21a-25e07dfba945 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:53.642665 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591446a-cd12-452f-b21a-25e07dfba945 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:53.680689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077c87e-378b-497e-8147-0c909f3c1d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:53.682191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077c87e-378b-497e-8147-0c909f3c1d8f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:53.837340 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 187/373] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:52:53.837910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591446a-cd12-452f-b21a-25e07dfba945 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:53.857263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:53.869608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 187/374] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:52:53.875568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:53.878155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077c87e-378b-497e-8147-0c909f3c1d8f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:53.965163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591446a-cd12-452f-b21a-25e07dfba945 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:54.044908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393f9a1-eec9-4670-a449-df5bee6b5bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:54.045710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393f9a1-eec9-4670-a449-df5bee6b5bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:54.046008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393f9a1-eec9-4670-a449-df5bee6b5bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:54.046286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393f9a1-eec9-4670-a449-df5bee6b5bf0 None admin] PATH_INFO: `/v3/projects/e2f95f7ebd0046e996236ac2938731ae` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:54.047994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393f9a1-eec9-4670-a449-df5bee6b5bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=e2f95f7ebd0046e996236ac2938731ae)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:54.049477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393f9a1-eec9-4670-a449-df5bee6b5bf0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:54.051459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 188/375] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:53 2022] GET /identity/v3/projects/e2f95f7ebd0046e996236ac2938731ae => generated 344 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:54.057883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393f9a1-eec9-4670-a449-df5bee6b5bf0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:54.058243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077c87e-378b-497e-8147-0c909f3c1d8f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:54.107123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b90ea-785e-42a2-aad1-a88907cb21c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:54.108093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b90ea-785e-42a2-aad1-a88907cb21c4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:54.108570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b90ea-785e-42a2-aad1-a88907cb21c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:54.109035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b90ea-785e-42a2-aad1-a88907cb21c4 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:54.112798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b90ea-785e-42a2-aad1-a88907cb21c4 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:54.114467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b90ea-785e-42a2-aad1-a88907cb21c4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:54.120788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 188/376] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:53 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:54.126921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b90ea-785e-42a2-aad1-a88907cb21c4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:54.197337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:54.229938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b90ea-785e-42a2-aad1-a88907cb21c4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:54.253464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:54.267937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046c71b-1af4-4424-ae19-91a035682b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:54.268814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046c71b-1af4-4424-ae19-91a035682b99 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:54.269326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046c71b-1af4-4424-ae19-91a035682b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:54.269768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046c71b-1af4-4424-ae19-91a035682b99 None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:54.271855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046c71b-1af4-4424-ae19-91a035682b99 None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:54.275168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393f9a1-eec9-4670-a449-df5bee6b5bf0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:54.275903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046c71b-1af4-4424-ae19-91a035682b99 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:54.278275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 189/377] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:52:54 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:54.288000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046c71b-1af4-4424-ae19-91a035682b99 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:54.356219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6ff3a-604b-4c94-8d83-5f99ef688514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:54.357282 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6ff3a-604b-4c94-8d83-5f99ef688514 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:54.358625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6ff3a-604b-4c94-8d83-5f99ef688514 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:54.361623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6ff3a-604b-4c94-8d83-5f99ef688514 None admin] PATH_INFO: `/v3/groups/1217606c342f417589bb5ab9445c5504` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:54.375701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6ff3a-604b-4c94-8d83-5f99ef688514 None admin] RBAC: Authorizing `identity:get_group(group_id=1217606c342f417589bb5ab9445c5504)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:54.377810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6ff3a-604b-4c94-8d83-5f99ef688514 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:54.385310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 189/378] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:52:54 2022] GET /identity/v3/groups/1217606c342f417589bb5ab9445c5504 => generated 229 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:54.388443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6ff3a-604b-4c94-8d83-5f99ef688514 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:54.488015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046c71b-1af4-4424-ae19-91a035682b99 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:54.537355 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:54.549665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffbe57-c5d6-4704-a28d-664238c6b256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:54.556420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffbe57-c5d6-4704-a28d-664238c6b256 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:54.556420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffbe57-c5d6-4704-a28d-664238c6b256 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:54.556420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffbe57-c5d6-4704-a28d-664238c6b256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:54.556420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffbe57-c5d6-4704-a28d-664238c6b256 None admin] RBAC: Adding query filter params (group.id=11ba0ae8b86745989af94117b91abdd3, role.id=f4c7e5a278e34f03a86f1f88ed77db0c, scope.project.id=1e2fa6632a6f4745b4a2406290657577) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:54.556420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffbe57-c5d6-4704-a28d-664238c6b256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:54.556420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffbe57-c5d6-4704-a28d-664238c6b256 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:54.565629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 190/379] 10.222.0.86 () {56 vars in 1376 bytes} [Thu Jul 28 17:52:54 2022] GET /identity/v3/role_assignments?group.id=11ba0ae8b86745989af94117b91abdd3&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=f4c7e5a278e34f03a86f1f88ed77db0c => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:54.588366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6ff3a-604b-4c94-8d83-5f99ef688514 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:54.631485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b0356-22b2-4aab-b9ec-902a9a3b27ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:54.632046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b0356-22b2-4aab-b9ec-902a9a3b27ee None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:54.632329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b0356-22b2-4aab-b9ec-902a9a3b27ee None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:54.632605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b0356-22b2-4aab-b9ec-902a9a3b27ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:54.633462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b0356-22b2-4aab-b9ec-902a9a3b27ee None admin] RBAC: Adding query filter params (group.id=1217606c342f417589bb5ab9445c5504, role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=e2f95f7ebd0046e996236ac2938731ae) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:54.633907 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b0356-22b2-4aab-b9ec-902a9a3b27ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:54.635373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b0356-22b2-4aab-b9ec-902a9a3b27ee None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:54.642743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 190/380] 10.222.0.86 () {56 vars in 1376 bytes} [Thu Jul 28 17:52:54 2022] GET /identity/v3/role_assignments?group.id=1217606c342f417589bb5ab9445c5504&scope.project.id=e2f95f7ebd0046e996236ac2938731ae&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 260 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:55.249296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec39651-91ef-437d-a748-1a34527d6034 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:55.250844 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec39651-91ef-437d-a748-1a34527d6034 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:55.251329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec39651-91ef-437d-a748-1a34527d6034 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:55.253471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 191/381] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:55.258846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ca641-83c8-421b-990c-5067b146ab2f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:55.259652 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ca641-83c8-421b-990c-5067b146ab2f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:55.260134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ca641-83c8-421b-990c-5067b146ab2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:55.283785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-144ca641-83c8-421b-990c-5067b146ab2f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:55.285339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-144ca641-83c8-421b-990c-5067b146ab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:55.308584 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:55.384151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fff08-34bd-45c7-a3a6-feca5f0be011 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:55.384518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fff08-34bd-45c7-a3a6-feca5f0be011 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:55.384724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fff08-34bd-45c7-a3a6-feca5f0be011 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:55.386396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 192/382] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:55.389532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 191/383] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:55.391719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d00d6-2802-4b7b-a1b6-bb17548cce99 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:55.392000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d00d6-2802-4b7b-a1b6-bb17548cce99 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:55.392272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d00d6-2802-4b7b-a1b6-bb17548cce99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:55.411240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c45d00d6-2802-4b7b-a1b6-bb17548cce99 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:55.412443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c45d00d6-2802-4b7b-a1b6-bb17548cce99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:55.424654 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ca641-83c8-421b-990c-5067b146ab2f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:55.516709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 193/384] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:55.565324 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d00d6-2802-4b7b-a1b6-bb17548cce99 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:55.605846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ca641-83c8-421b-990c-5067b146ab2f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:55.657992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82587e85-29f5-4286-a247-51e1be2a8651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:55.658685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587e85-29f5-4286-a247-51e1be2a8651 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:55.658971 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587e85-29f5-4286-a247-51e1be2a8651 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:55.659260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587e85-29f5-4286-a247-51e1be2a8651 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:55.670579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82587e85-29f5-4286-a247-51e1be2a8651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-82587e85-29f5-4286-a247-51e1be2a8651 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:55.675798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:55.676686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:55.677532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 192/385] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:55.678653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce404af0-078c-49d0-9693-65c6ce22333b None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:55.678815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce404af0-078c-49d0-9693-65c6ce22333b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:55.679069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce404af0-078c-49d0-9693-65c6ce22333b None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:55.684326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 193/386] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:55.684326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce404af0-078c-49d0-9693-65c6ce22333b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:55.695210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:55.721122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d00d6-2802-4b7b-a1b6-bb17548cce99 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:55.753241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:55.771105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd04852-f775-419d-aa8d-68e2641b2781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:55.771737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd04852-f775-419d-aa8d-68e2641b2781 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:55.772074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd04852-f775-419d-aa8d-68e2641b2781 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:55.772358 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd04852-f775-419d-aa8d-68e2641b2781 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:55.783461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd04852-f775-419d-aa8d-68e2641b2781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8bd04852-f775-419d-aa8d-68e2641b2781 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:55.789549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:55.790446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:55.791266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:55.795836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 194/387] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:55.797187 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f69d0-ba5c-455c-9602-ca7c689e304a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:55.797482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f69d0-ba5c-455c-9602-ca7c689e304a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:55.797768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f69d0-ba5c-455c-9602-ca7c689e304a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:55.809554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce404af0-078c-49d0-9693-65c6ce22333b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:55.816167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f48f69d0-ba5c-455c-9602-ca7c689e304a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:55.817420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f48f69d0-ba5c-455c-9602-ca7c689e304a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:55.850328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c1a8a-913c-4605-b51b-bcbb06810c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:55.850843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c1a8a-913c-4605-b51b-bcbb06810c01 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:55.851131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c1a8a-913c-4605-b51b-bcbb06810c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:55.851411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c1a8a-913c-4605-b51b-bcbb06810c01 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:55.852304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c1a8a-913c-4605-b51b-bcbb06810c01 None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:55.852620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c1a8a-913c-4605-b51b-bcbb06810c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:55.854192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c1a8a-913c-4605-b51b-bcbb06810c01 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:55.863414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 194/388] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:55.864324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c1a8a-913c-4605-b51b-bcbb06810c01 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:55.909084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 195/389] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:55.910269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f69d0-ba5c-455c-9602-ca7c689e304a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:55.941366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:55.979633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c1a8a-913c-4605-b51b-bcbb06810c01 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:56.017165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383139e6-8cbb-4742-ba0f-f2c4097feeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:56.017795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383139e6-8cbb-4742-ba0f-f2c4097feeed None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:56.018076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383139e6-8cbb-4742-ba0f-f2c4097feeed None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:56.018656 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383139e6-8cbb-4742-ba0f-f2c4097feeed None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:56.019179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383139e6-8cbb-4742-ba0f-f2c4097feeed None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:56.019488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383139e6-8cbb-4742-ba0f-f2c4097feeed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:56.020980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383139e6-8cbb-4742-ba0f-f2c4097feeed None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:56.030137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 195/390] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:56.030871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383139e6-8cbb-4742-ba0f-f2c4097feeed None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:56.042779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f69d0-ba5c-455c-9602-ca7c689e304a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:56.057998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:56.080547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d92919-d0c2-4f88-a25c-ba23c6ef0851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:56.081666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d92919-d0c2-4f88-a25c-ba23c6ef0851 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:56.082172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d92919-d0c2-4f88-a25c-ba23c6ef0851 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:56.082650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d92919-d0c2-4f88-a25c-ba23c6ef0851 None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:56.084132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d92919-d0c2-4f88-a25c-ba23c6ef0851 None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:56.085968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d92919-d0c2-4f88-a25c-ba23c6ef0851 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:56.087910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 196/391] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:52:55 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:56.088855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d92919-d0c2-4f88-a25c-ba23c6ef0851 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:56.152133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383139e6-8cbb-4742-ba0f-f2c4097feeed None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:56.182594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:56.189404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a309f4-a3c8-4c69-a25c-715c0c6b71b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:56.190058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a309f4-a3c8-4c69-a25c-715c0c6b71b5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:56.190489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a309f4-a3c8-4c69-a25c-715c0c6b71b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:56.190711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a309f4-a3c8-4c69-a25c-715c0c6b71b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:56.200653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d92919-d0c2-4f88-a25c-ba23c6ef0851 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:56.204414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a309f4-a3c8-4c69-a25c-715c0c6b71b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-55a309f4-a3c8-4c69-a25c-715c0c6b71b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:56.209463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:56.210345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:56.211137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:56.215510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 196/392] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:52:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:56.216470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a309f4-a3c8-4c69-a25c-715c0c6b71b5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:56.241155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181cf99-05a2-49ae-abeb-3a56968eed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:56.242138 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cf99-05a2-49ae-abeb-3a56968eed4d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:56.242720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cf99-05a2-49ae-abeb-3a56968eed4d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:56.243221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cf99-05a2-49ae-abeb-3a56968eed4d None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:56.244626 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181cf99-05a2-49ae-abeb-3a56968eed4d None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:56.246517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181cf99-05a2-49ae-abeb-3a56968eed4d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:56.248679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 197/393] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:52:56 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:56.250337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181cf99-05a2-49ae-abeb-3a56968eed4d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:56.304625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:56.327153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a309f4-a3c8-4c69-a25c-715c0c6b71b5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:56.362279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181cf99-05a2-49ae-abeb-3a56968eed4d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:56.363784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428531-dfd8-4882-ab50-4d6a0b013d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:56.364842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428531-dfd8-4882-ab50-4d6a0b013d58 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:56.365403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428531-dfd8-4882-ab50-4d6a0b013d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:56.365896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428531-dfd8-4882-ab50-4d6a0b013d58 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:56.367302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95428531-dfd8-4882-ab50-4d6a0b013d58 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:56.369105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95428531-dfd8-4882-ab50-4d6a0b013d58 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:56.376730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 197/394] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:56 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:56.378020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428531-dfd8-4882-ab50-4d6a0b013d58 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:56.399383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c95d0-7e25-4a1d-abf4-8f5e6b82c1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:56.400331 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c95d0-7e25-4a1d-abf4-8f5e6b82c1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:56.400769 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c95d0-7e25-4a1d-abf4-8f5e6b82c1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:56.401220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c95d0-7e25-4a1d-abf4-8f5e6b82c1c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:56.402187 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c95d0-7e25-4a1d-abf4-8f5e6b82c1c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:56.402701 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c95d0-7e25-4a1d-abf4-8f5e6b82c1c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:56.404293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c95d0-7e25-4a1d-abf4-8f5e6b82c1c7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:56.420194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 198/395] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:52:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:56.422456 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c95d0-7e25-4a1d-abf4-8f5e6b82c1c7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:56.452611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:56.496414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428531-dfd8-4882-ab50-4d6a0b013d58 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:56.530607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c95d0-7e25-4a1d-abf4-8f5e6b82c1c7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:56.532204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad867-cef5-4804-8c65-ad1294a3288d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:56.533133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad867-cef5-4804-8c65-ad1294a3288d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:56.533622 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad867-cef5-4804-8c65-ad1294a3288d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:56.534126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad867-cef5-4804-8c65-ad1294a3288d None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:56.535524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ad867-cef5-4804-8c65-ad1294a3288d None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:56.537188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ad867-cef5-4804-8c65-ad1294a3288d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:56.539371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 198/396] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:56 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:56.540406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad867-cef5-4804-8c65-ad1294a3288d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:56.545023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:56.568231 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109deea-a14c-4f95-958f-9b443254423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:56.568867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109deea-a14c-4f95-958f-9b443254423d None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:56.569182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109deea-a14c-4f95-958f-9b443254423d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:56.569476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109deea-a14c-4f95-958f-9b443254423d None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577/groups/11ba0ae8b86745989af94117b91abdd3/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:56.571559 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109deea-a14c-4f95-958f-9b443254423d None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2fa6632a6f4745b4a2406290657577, group_id=11ba0ae8b86745989af94117b91abdd3, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:56.573231 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109deea-a14c-4f95-958f-9b443254423d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:56.647182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad867-cef5-4804-8c65-ad1294a3288d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:56.649548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 199/397] 10.222.0.86 () {58 vars in 1442 bytes} [Thu Jul 28 17:52:56 2022] PUT /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577/groups/11ba0ae8b86745989af94117b91abdd3/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:56.650410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109deea-a14c-4f95-958f-9b443254423d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:56.681617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:56.686791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b0605-ccdb-4f28-b60d-98f53d48dfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:56.687711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b0605-ccdb-4f28-b60d-98f53d48dfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:56.688215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b0605-ccdb-4f28-b60d-98f53d48dfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:56.688691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b0605-ccdb-4f28-b60d-98f53d48dfe0 None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:56.690120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b0605-ccdb-4f28-b60d-98f53d48dfe0 None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:56.697560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b0605-ccdb-4f28-b60d-98f53d48dfe0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:56.697560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 199/398] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:52:56 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:56.704747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b0605-ccdb-4f28-b60d-98f53d48dfe0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:56.832016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b0605-ccdb-4f28-b60d-98f53d48dfe0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:56.874223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1d373-6d09-4ac9-ab75-c4c5a10e0eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:56.875403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1d373-6d09-4ac9-ab75-c4c5a10e0eaa None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:56.875892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1d373-6d09-4ac9-ab75-c4c5a10e0eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:56.876778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1d373-6d09-4ac9-ab75-c4c5a10e0eaa None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:56.878657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1d373-6d09-4ac9-ab75-c4c5a10e0eaa None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:56.880446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1d373-6d09-4ac9-ab75-c4c5a10e0eaa None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:56.883662 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 200/399] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:56 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:56.888125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109deea-a14c-4f95-958f-9b443254423d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:56.890594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1d373-6d09-4ac9-ab75-c4c5a10e0eaa None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:56.935020 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69231e-590b-4305-ac8a-45547395b483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:56.936659 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69231e-590b-4305-ac8a-45547395b483 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:56.937545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69231e-590b-4305-ac8a-45547395b483 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:56.937915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69231e-590b-4305-ac8a-45547395b483 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0/groups/11ba0ae8b86745989af94117b91abdd3/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:56.940157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69231e-590b-4305-ac8a-45547395b483 None admin] RBAC: Authorizing `identity:create_grant(project_id=de970216eec1471ca4c15bf6365e4ef0, group_id=11ba0ae8b86745989af94117b91abdd3, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:56.941799 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69231e-590b-4305-ac8a-45547395b483 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:56.969856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:57.015728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1d373-6d09-4ac9-ab75-c4c5a10e0eaa None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:57.046392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 200/400] 10.222.0.86 () {58 vars in 1442 bytes} [Thu Jul 28 17:52:56 2022] PUT /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0/groups/11ba0ae8b86745989af94117b91abdd3/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:57.048448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535381e8-709f-41e3-ae07-8c7108a6b4c5 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:57.048754 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535381e8-709f-41e3-ae07-8c7108a6b4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:57.049054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535381e8-709f-41e3-ae07-8c7108a6b4c5 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:57.050444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 201/401] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:57.054703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bb758-56ad-41fa-8957-175017f8e782 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:57.054703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bb758-56ad-41fa-8957-175017f8e782 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:57.054703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bb758-56ad-41fa-8957-175017f8e782 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:57.058397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 202/402] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:57.059926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9be7f-acc1-46cf-a251-2b7db42df2ea None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:57.060217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9be7f-acc1-46cf-a251-2b7db42df2ea None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:57.060493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9be7f-acc1-46cf-a251-2b7db42df2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:57.062005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa6d86-5e96-4137-8560-39d08730a4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:57.063020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa6d86-5e96-4137-8560-39d08730a4d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:57.063630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa6d86-5e96-4137-8560-39d08730a4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:57.064113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa6d86-5e96-4137-8560-39d08730a4d8 None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0/groups/11ba0ae8b86745989af94117b91abdd3/roles/f4c7e5a278e34f03a86f1f88ed77db0c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:57.066886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa6d86-5e96-4137-8560-39d08730a4d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=de970216eec1471ca4c15bf6365e4ef0, group_id=11ba0ae8b86745989af94117b91abdd3, role_id=f4c7e5a278e34f03a86f1f88ed77db0c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:57.069253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa6d86-5e96-4137-8560-39d08730a4d8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:57.081446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f3f9be7f-acc1-46cf-a251-2b7db42df2ea None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:57.082858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f3f9be7f-acc1-46cf-a251-2b7db42df2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:57.112633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:57.158622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 203/404] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:57.159015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 201/404] 10.222.0.86 () {58 vars in 1442 bytes} [Thu Jul 28 17:52:56 2022] PUT /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0/groups/11ba0ae8b86745989af94117b91abdd3/roles/f4c7e5a278e34f03a86f1f88ed77db0c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:57.160201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ce820-d132-466d-bd34-bb94a437f237 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:57.160407 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ce820-d132-466d-bd34-bb94a437f237 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:57.160697 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ce820-d132-466d-bd34-bb94a437f237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:57.181609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d21ce820-d132-466d-bd34-bb94a437f237 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:57.183046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d21ce820-d132-466d-bd34-bb94a437f237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:57.208982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa6d86-5e96-4137-8560-39d08730a4d8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:57.265131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 204/405] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:57.294290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ce820-d132-466d-bd34-bb94a437f237 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:57.399690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:57.413231 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ce820-d132-466d-bd34-bb94a437f237 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:57.443132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa6d86-5e96-4137-8560-39d08730a4d8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:57.473068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:57.474665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748307bf-063a-4b0e-955f-73e08935b258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:57.475593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748307bf-063a-4b0e-955f-73e08935b258 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:57.475873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748307bf-063a-4b0e-955f-73e08935b258 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:57.476146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748307bf-063a-4b0e-955f-73e08935b258 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:57.489707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7de9bb-f423-4114-8199-7a62449cf0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:57.490367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7de9bb-f423-4114-8199-7a62449cf0bf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:57.490649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7de9bb-f423-4114-8199-7a62449cf0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:57.491757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7de9bb-f423-4114-8199-7a62449cf0bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:57.493464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748307bf-063a-4b0e-955f-73e08935b258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-748307bf-063a-4b0e-955f-73e08935b258 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:57.500348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:57.501255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:52:57.502385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:52:57.505168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 205/406] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:52:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:57.507806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7de9bb-f423-4114-8199-7a62449cf0bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:57.508652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748307bf-063a-4b0e-955f-73e08935b258 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ed7de9bb-f423-4114-8199-7a62449cf0bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:57.516393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:57.519504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:57.520891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 202/407] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:52:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:57.526175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7de9bb-f423-4114-8199-7a62449cf0bf None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:57.635532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7de9bb-f423-4114-8199-7a62449cf0bf None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:57.673642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6163638-7cc6-430c-a2eb-8a2ffc3167cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:57.674187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6163638-7cc6-430c-a2eb-8a2ffc3167cb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:57.674467 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6163638-7cc6-430c-a2eb-8a2ffc3167cb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:57.674738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6163638-7cc6-430c-a2eb-8a2ffc3167cb None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:57.675460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6163638-7cc6-430c-a2eb-8a2ffc3167cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:57.675763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6163638-7cc6-430c-a2eb-8a2ffc3167cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:57.677151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6163638-7cc6-430c-a2eb-8a2ffc3167cb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:57.681930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748307bf-063a-4b0e-955f-73e08935b258 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:57.697984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 203/408] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:52:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:57.704208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6163638-7cc6-430c-a2eb-8a2ffc3167cb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:57.747000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1c85e-87af-4867-99cd-8d5c6fb27939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:57.747974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1c85e-87af-4867-99cd-8d5c6fb27939 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:57.748498 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1c85e-87af-4867-99cd-8d5c6fb27939 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:57.749190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1c85e-87af-4867-99cd-8d5c6fb27939 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:57.751641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1c85e-87af-4867-99cd-8d5c6fb27939 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:57.752197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1c85e-87af-4867-99cd-8d5c6fb27939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:57.753927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1c85e-87af-4867-99cd-8d5c6fb27939 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:57.772055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 206/409] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:57.778653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1c85e-87af-4867-99cd-8d5c6fb27939 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:57.797868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:57.810718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:57.897009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6163638-7cc6-430c-a2eb-8a2ffc3167cb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:57.965764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8de93-7da9-444b-a657-29b81b7301f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:57.966653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8de93-7da9-444b-a657-29b81b7301f8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:57.968540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8de93-7da9-444b-a657-29b81b7301f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:57.968989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8de93-7da9-444b-a657-29b81b7301f8 None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:57.972116 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8de93-7da9-444b-a657-29b81b7301f8 None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:57.977016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1c85e-87af-4867-99cd-8d5c6fb27939 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:57.978450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8de93-7da9-444b-a657-29b81b7301f8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:57.982268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 204/410] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:52:57 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:57.989139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8de93-7da9-444b-a657-29b81b7301f8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:58.027653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153191d-7a8c-4be3-b984-1fefcb9ea5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:58.028539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153191d-7a8c-4be3-b984-1fefcb9ea5ee None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:58.028989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153191d-7a8c-4be3-b984-1fefcb9ea5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:58.029410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153191d-7a8c-4be3-b984-1fefcb9ea5ee None admin] PATH_INFO: `/v3/groups/1217606c342f417589bb5ab9445c5504` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:58.030756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153191d-7a8c-4be3-b984-1fefcb9ea5ee None admin] RBAC: Authorizing `identity:get_group(group_id=1217606c342f417589bb5ab9445c5504)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:58.032373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153191d-7a8c-4be3-b984-1fefcb9ea5ee None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:58.034487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 207/411] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:52:57 2022] GET /identity/v3/groups/1217606c342f417589bb5ab9445c5504 => generated 229 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:58.040012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153191d-7a8c-4be3-b984-1fefcb9ea5ee None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:58.171632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:58.184773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8de93-7da9-444b-a657-29b81b7301f8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:58.191689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153191d-7a8c-4be3-b984-1fefcb9ea5ee None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:58.211900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:58.228524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e40ba5-d99e-4a77-9517-2507a507f02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:58.229133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40ba5-d99e-4a77-9517-2507a507f02e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:58.229420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40ba5-d99e-4a77-9517-2507a507f02e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:58.229698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40ba5-d99e-4a77-9517-2507a507f02e None admin] PATH_INFO: `/v3/projects/e2f95f7ebd0046e996236ac2938731ae` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:58.231143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e40ba5-d99e-4a77-9517-2507a507f02e None admin] RBAC: Authorizing `identity:get_project(project_id=e2f95f7ebd0046e996236ac2938731ae)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:58.232710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e40ba5-d99e-4a77-9517-2507a507f02e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:58.234846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 208/412] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:58 2022] GET /identity/v3/projects/e2f95f7ebd0046e996236ac2938731ae => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:58.242618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e40ba5-d99e-4a77-9517-2507a507f02e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:58.268600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1de91-b298-4763-a020-0b4e534725ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:58.269725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1de91-b298-4763-a020-0b4e534725ed None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:58.270012 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1de91-b298-4763-a020-0b4e534725ed None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:58.270292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1de91-b298-4763-a020-0b4e534725ed None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:58.274539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1de91-b298-4763-a020-0b4e534725ed None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:58.278505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1de91-b298-4763-a020-0b4e534725ed None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:58.282704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 205/413] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:52:57 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:58.287439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1de91-b298-4763-a020-0b4e534725ed None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:58.419013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1de91-b298-4763-a020-0b4e534725ed None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:58.449439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e40ba5-d99e-4a77-9517-2507a507f02e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:58.457886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016fd46-1e9f-420a-b110-369d519a0b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:58.458576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016fd46-1e9f-420a-b110-369d519a0b09 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:58.458862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016fd46-1e9f-420a-b110-369d519a0b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:58.459141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016fd46-1e9f-420a-b110-369d519a0b09 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577/groups/11ba0ae8b86745989af94117b91abdd3/roles/f4c7e5a278e34f03a86f1f88ed77db0c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:58.463481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016fd46-1e9f-420a-b110-369d519a0b09 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2fa6632a6f4745b4a2406290657577, group_id=11ba0ae8b86745989af94117b91abdd3, role_id=f4c7e5a278e34f03a86f1f88ed77db0c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:58.463481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016fd46-1e9f-420a-b110-369d519a0b09 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:58.502192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccf710-ad30-4d70-8743-37c0e1222b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:58.503247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf710-ad30-4d70-8743-37c0e1222b17 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:58.503753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf710-ad30-4d70-8743-37c0e1222b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:58.504295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccf710-ad30-4d70-8743-37c0e1222b17 None admin] PATH_INFO: `/v3/projects/e2f95f7ebd0046e996236ac2938731ae/groups/1217606c342f417589bb5ab9445c5504/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:58.506619 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccf710-ad30-4d70-8743-37c0e1222b17 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2f95f7ebd0046e996236ac2938731ae, group_id=1217606c342f417589bb5ab9445c5504, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:58.508409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccf710-ad30-4d70-8743-37c0e1222b17 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:58.540428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 206/414] 10.222.0.86 () {58 vars in 1442 bytes} [Thu Jul 28 17:52:58 2022] PUT /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577/groups/11ba0ae8b86745989af94117b91abdd3/roles/f4c7e5a278e34f03a86f1f88ed77db0c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:52:58.575417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 209/415] 10.222.0.86 () {58 vars in 1442 bytes} [Thu Jul 28 17:52:58 2022] PUT /identity/v3/projects/e2f95f7ebd0046e996236ac2938731ae/groups/1217606c342f417589bb5ab9445c5504/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:59.058255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516564a-821c-449b-926b-fb055aa9c03e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:59.058621 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516564a-821c-449b-926b-fb055aa9c03e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:59.058821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516564a-821c-449b-926b-fb055aa9c03e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:59.060453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 207/416] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:59.066257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec02d4-290b-4057-bbe3-603c02068ffe None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:59.078433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec02d4-290b-4057-bbe3-603c02068ffe None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:59.078433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec02d4-290b-4057-bbe3-603c02068ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:59.091848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0aec02d4-290b-4057-bbe3-603c02068ffe None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:59.093062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0aec02d4-290b-4057-bbe3-603c02068ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:59.096091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:59.182657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 210/417] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:52:59.219604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516564a-821c-449b-926b-fb055aa9c03e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:59.274379 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:59.342686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516564a-821c-449b-926b-fb055aa9c03e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:59.393541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e234c57-93cb-4422-8158-3dea7754643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:59.394209 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e234c57-93cb-4422-8158-3dea7754643e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:59.394495 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e234c57-93cb-4422-8158-3dea7754643e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:59.394776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e234c57-93cb-4422-8158-3dea7754643e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:59.405685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e234c57-93cb-4422-8158-3dea7754643e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-4e234c57-93cb-4422-8158-3dea7754643e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:59.411977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:59.412891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:59.413735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 208/418] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:52:59.429466 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec02d4-290b-4057-bbe3-603c02068ffe None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:59.455001 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfcdfc-c173-4965-9d88-105ab551f99f None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:59.455292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfcdfc-c173-4965-9d88-105ab551f99f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:59.455568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfcdfc-c173-4965-9d88-105ab551f99f None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:59.457223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 209/419] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:52:59.461917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471ccf4-e5c3-4f20-adce-0c7f0c3dc94e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:59.462439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471ccf4-e5c3-4f20-adce-0c7f0c3dc94e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:59.462511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471ccf4-e5c3-4f20-adce-0c7f0c3dc94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:59.497099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a471ccf4-e5c3-4f20-adce-0c7f0c3dc94e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:59.498353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a471ccf4-e5c3-4f20-adce-0c7f0c3dc94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:59.583692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec02d4-290b-4057-bbe3-603c02068ffe None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:59.642190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 210/420] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:52:59.665663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610672f-6852-4224-b541-f17d444fe119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:59.666298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610672f-6852-4224-b541-f17d444fe119 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:59.666586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610672f-6852-4224-b541-f17d444fe119 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:59.666947 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610672f-6852-4224-b541-f17d444fe119 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:59.669474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610672f-6852-4224-b541-f17d444fe119 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:59.669474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610672f-6852-4224-b541-f17d444fe119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:59.669474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610672f-6852-4224-b541-f17d444fe119 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:59.673866 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471ccf4-e5c3-4f20-adce-0c7f0c3dc94e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:59.679179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 211/421] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:52:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:59.688011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610672f-6852-4224-b541-f17d444fe119 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:59.697282 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:59.736517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:59.819020 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610672f-6852-4224-b541-f17d444fe119 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:59.846535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471ccf4-e5c3-4f20-adce-0c7f0c3dc94e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:59.860742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012041ce-a5ee-4906-b7b1-622f5d4cc8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:59.861759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012041ce-a5ee-4906-b7b1-622f5d4cc8ab None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:59.862186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012041ce-a5ee-4906-b7b1-622f5d4cc8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:59.862584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012041ce-a5ee-4906-b7b1-622f5d4cc8ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:59.882055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c09d8-4ac9-451a-9235-3f2e5b4c57c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:59.882838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c09d8-4ac9-451a-9235-3f2e5b4c57c1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:59.883226 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c09d8-4ac9-451a-9235-3f2e5b4c57c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:59.883604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c09d8-4ac9-451a-9235-3f2e5b4c57c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:59.898729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c09d8-4ac9-451a-9235-3f2e5b4c57c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-aa6c09d8-4ac9-451a-9235-3f2e5b4c57c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:52:59.903609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:52:59.904529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:52:59.905377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 211/422] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:52:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:52:59.906805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98f3d0-fc77-430f-872e-ea09e6cf059a None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:59.907447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98f3d0-fc77-430f-872e-ea09e6cf059a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:59.907878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98f3d0-fc77-430f-872e-ea09e6cf059a None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:59.909354 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 212/423] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:52:59.915068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da856b8e-b9f5-4471-8f69-50ffd4e473bf None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:59.915466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da856b8e-b9f5-4471-8f69-50ffd4e473bf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:59.915846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da856b8e-b9f5-4471-8f69-50ffd4e473bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:59.934042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-da856b8e-b9f5-4471-8f69-50ffd4e473bf None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:59.935384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-da856b8e-b9f5-4471-8f69-50ffd4e473bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:59.950867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012041ce-a5ee-4906-b7b1-622f5d4cc8ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:59.952127 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012041ce-a5ee-4906-b7b1-622f5d4cc8ab None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:59.957388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:00.016295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 213/424] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:00.018076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da856b8e-b9f5-4471-8f69-50ffd4e473bf None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:00.044846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:00.068232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 212/425] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:53:00.069410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012041ce-a5ee-4906-b7b1-622f5d4cc8ab None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:00.127181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da856b8e-b9f5-4471-8f69-50ffd4e473bf None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:00.164677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccf426-95d7-41e5-a3ab-453dcebefecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:00.165424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf426-95d7-41e5-a3ab-453dcebefecd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:00.165807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf426-95d7-41e5-a3ab-453dcebefecd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:00.166179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf426-95d7-41e5-a3ab-453dcebefecd None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:00.167084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccf426-95d7-41e5-a3ab-453dcebefecd None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:00.167640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccf426-95d7-41e5-a3ab-453dcebefecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:00.169196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccf426-95d7-41e5-a3ab-453dcebefecd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:00.180017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 214/426] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:53:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:00.181050 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccf426-95d7-41e5-a3ab-453dcebefecd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:00.192940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012041ce-a5ee-4906-b7b1-622f5d4cc8ab None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:00.201252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:00.230129 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea961436-cabd-4acb-bcfd-eb0a4a55bd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:00.230772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea961436-cabd-4acb-bcfd-eb0a4a55bd11 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:00.231063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea961436-cabd-4acb-bcfd-eb0a4a55bd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:00.231387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea961436-cabd-4acb-bcfd-eb0a4a55bd11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:00.242625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea961436-cabd-4acb-bcfd-eb0a4a55bd11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ea961436-cabd-4acb-bcfd-eb0a4a55bd11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:00.248104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:00.249206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 213/427] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:00.250150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea961436-cabd-4acb-bcfd-eb0a4a55bd11 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:00.268763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:00.287483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccf426-95d7-41e5-a3ab-453dcebefecd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:00.321553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c71539-ee4c-41d9-9e22-d3764e0a8791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:00.322322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c71539-ee4c-41d9-9e22-d3764e0a8791 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:00.322703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c71539-ee4c-41d9-9e22-d3764e0a8791 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:00.323074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c71539-ee4c-41d9-9e22-d3764e0a8791 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:00.324339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c71539-ee4c-41d9-9e22-d3764e0a8791 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:00.325899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c71539-ee4c-41d9-9e22-d3764e0a8791 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:00.328177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 215/428] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:53:00 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:00.329496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c71539-ee4c-41d9-9e22-d3764e0a8791 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:00.364606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea961436-cabd-4acb-bcfd-eb0a4a55bd11 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:00.403310 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feb0ed-3dc9-4679-85ec-4d14c9c4eec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:00.404320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feb0ed-3dc9-4679-85ec-4d14c9c4eec4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:00.404749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feb0ed-3dc9-4679-85ec-4d14c9c4eec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:00.405224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feb0ed-3dc9-4679-85ec-4d14c9c4eec4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:00.435293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c71539-ee4c-41d9-9e22-d3764e0a8791 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:00.450233 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:00.471874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5d3ab-5873-4d63-ab18-eb40a9f8bcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:00.472674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5d3ab-5873-4d63-ab18-eb40a9f8bcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:00.473099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5d3ab-5873-4d63-ab18-eb40a9f8bcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:00.473457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5d3ab-5873-4d63-ab18-eb40a9f8bcd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:00.474435 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5d3ab-5873-4d63-ab18-eb40a9f8bcd9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:00.474835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5d3ab-5873-4d63-ab18-eb40a9f8bcd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:00.476307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5d3ab-5873-4d63-ab18-eb40a9f8bcd9 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:00.485256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 216/429] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:00.486252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5d3ab-5873-4d63-ab18-eb40a9f8bcd9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:00.496261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97feb0ed-3dc9-4679-85ec-4d14c9c4eec4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:00.499382 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97feb0ed-3dc9-4679-85ec-4d14c9c4eec4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:00.509742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:00.616269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 214/430] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:00.616269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feb0ed-3dc9-4679-85ec-4d14c9c4eec4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:00.630256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5d3ab-5873-4d63-ab18-eb40a9f8bcd9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:00.677738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6108f11-2e30-4f8a-b7fd-fbcffd9e23fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:00.679063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6108f11-2e30-4f8a-b7fd-fbcffd9e23fd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:00.679505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6108f11-2e30-4f8a-b7fd-fbcffd9e23fd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:00.680176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6108f11-2e30-4f8a-b7fd-fbcffd9e23fd None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:00.681815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6108f11-2e30-4f8a-b7fd-fbcffd9e23fd None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:00.684084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6108f11-2e30-4f8a-b7fd-fbcffd9e23fd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:00.684723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:00.686835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 217/431] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:53:00 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:00.687983 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6108f11-2e30-4f8a-b7fd-fbcffd9e23fd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:00.732010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feb0ed-3dc9-4679-85ec-4d14c9c4eec4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:00.771695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aadc85-436d-4c4b-8500-54ec902115d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:00.772343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aadc85-436d-4c4b-8500-54ec902115d0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:00.772631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aadc85-436d-4c4b-8500-54ec902115d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:00.772911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aadc85-436d-4c4b-8500-54ec902115d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:00.801585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:00.828387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6108f11-2e30-4f8a-b7fd-fbcffd9e23fd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:00.848777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aadc85-436d-4c4b-8500-54ec902115d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:00.850471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aadc85-436d-4c4b-8500-54ec902115d0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:00.874822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820b84b-fa5c-42c6-8fe4-deb22065a05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:00.876874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820b84b-fa5c-42c6-8fe4-deb22065a05d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:00.877446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820b84b-fa5c-42c6-8fe4-deb22065a05d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:00.878186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820b84b-fa5c-42c6-8fe4-deb22065a05d None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:00.880318 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820b84b-fa5c-42c6-8fe4-deb22065a05d None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:00.882999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820b84b-fa5c-42c6-8fe4-deb22065a05d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:00.885620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 218/432] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:53:00 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:00.891132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820b84b-fa5c-42c6-8fe4-deb22065a05d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:00.925102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:00.977076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 215/433] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:00.978958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aadc85-436d-4c4b-8500-54ec902115d0 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:01.011335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820b84b-fa5c-42c6-8fe4-deb22065a05d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:01.045518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751d225-4f5c-4360-ad79-7dd31286735e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:01.046133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751d225-4f5c-4360-ad79-7dd31286735e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.046442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751d225-4f5c-4360-ad79-7dd31286735e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.046720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751d225-4f5c-4360-ad79-7dd31286735e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.047511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751d225-4f5c-4360-ad79-7dd31286735e None admin] RBAC: Adding query filter params (group.id=11ba0ae8b86745989af94117b91abdd3, role.id=2dc5251f14e945e38ad1777a5f235180, scope.project.id=1e2fa6632a6f4745b4a2406290657577) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:01.047825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751d225-4f5c-4360-ad79-7dd31286735e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:01.049277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751d225-4f5c-4360-ad79-7dd31286735e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:01.054921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 219/434] 10.222.0.86 () {56 vars in 1376 bytes} [Thu Jul 28 17:53:00 2022] GET /identity/v3/role_assignments?group.id=11ba0ae8b86745989af94117b91abdd3&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:01.056063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c03682-17fa-4714-81dd-18ff34ea28dd None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.056350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c03682-17fa-4714-81dd-18ff34ea28dd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.057119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c03682-17fa-4714-81dd-18ff34ea28dd None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.058024 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 220/435] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:01.062728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a333f-5248-4b62-83e5-cbef7a9f43c4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.063024 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a333f-5248-4b62-83e5-cbef7a9f43c4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.063314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a333f-5248-4b62-83e5-cbef7a9f43c4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.065374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 221/436] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:01.066005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a333f-5248-4b62-83e5-cbef7a9f43c4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:01.082812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aadc85-436d-4c4b-8500-54ec902115d0 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:01.083290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:01.127714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16475072-5b40-49ca-a78d-739dce89fcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:01.128559 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16475072-5b40-49ca-a78d-739dce89fcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.128782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16475072-5b40-49ca-a78d-739dce89fcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.129098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16475072-5b40-49ca-a78d-739dce89fcf0 None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.130832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16475072-5b40-49ca-a78d-739dce89fcf0 None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:01.131993 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16475072-5b40-49ca-a78d-739dce89fcf0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:01.138666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 216/437] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:53:00 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:01.138666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa71fe0-126a-4795-8121-0708741fa477 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.138666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa71fe0-126a-4795-8121-0708741fa477 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.138666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa71fe0-126a-4795-8121-0708741fa477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.157313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-dfa71fe0-126a-4795-8121-0708741fa477 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:01.158570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-dfa71fe0-126a-4795-8121-0708741fa477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:01.189819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a333f-5248-4b62-83e5-cbef7a9f43c4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:01.191842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:01.236582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1db8d2-c015-4c4e-9891-af6a539b1f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:01.237199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1db8d2-c015-4c4e-9891-af6a539b1f4f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.237481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1db8d2-c015-4c4e-9891-af6a539b1f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.237757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1db8d2-c015-4c4e-9891-af6a539b1f4f None admin] PATH_INFO: `/v3/projects/de970216eec1471ca4c15bf6365e4ef0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.239096 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1db8d2-c015-4c4e-9891-af6a539b1f4f None admin] RBAC: Authorizing `identity:get_project(project_id=de970216eec1471ca4c15bf6365e4ef0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:01.240502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1db8d2-c015-4c4e-9891-af6a539b1f4f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:01.241108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 217/438] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:01.242291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d315f57-83b0-47de-b29c-a4cce0c34384 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.242400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 222/439] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:53:01 2022] GET /identity/v3/projects/de970216eec1471ca4c15bf6365e4ef0 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:01.242499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d315f57-83b0-47de-b29c-a4cce0c34384 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.242746 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d315f57-83b0-47de-b29c-a4cce0c34384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.243069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1db8d2-c015-4c4e-9891-af6a539b1f4f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:01.268765 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8d315f57-83b0-47de-b29c-a4cce0c34384 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:01.270380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8d315f57-83b0-47de-b29c-a4cce0c34384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:01.337745 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:01.339766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 218/440] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:01.340753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d315f57-83b0-47de-b29c-a4cce0c34384 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:01.356843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1db8d2-c015-4c4e-9891-af6a539b1f4f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:01.392723 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082731d4-c80c-4f2a-8096-7789f1bce4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:01.393370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082731d4-c80c-4f2a-8096-7789f1bce4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.393654 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082731d4-c80c-4f2a-8096-7789f1bce4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.393932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082731d4-c80c-4f2a-8096-7789f1bce4b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.394726 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082731d4-c80c-4f2a-8096-7789f1bce4b8 None admin] RBAC: Adding query filter params (group.id=11ba0ae8b86745989af94117b91abdd3, role.id=2dc5251f14e945e38ad1777a5f235180, scope.project.id=de970216eec1471ca4c15bf6365e4ef0) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:01.395035 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082731d4-c80c-4f2a-8096-7789f1bce4b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:01.396481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082731d4-c80c-4f2a-8096-7789f1bce4b8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:01.402264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 223/441] 10.222.0.86 () {56 vars in 1376 bytes} [Thu Jul 28 17:53:01 2022] GET /identity/v3/role_assignments?group.id=11ba0ae8b86745989af94117b91abdd3&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=2dc5251f14e945e38ad1777a5f235180 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:01.403227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082731d4-c80c-4f2a-8096-7789f1bce4b8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:01.437499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:01.453951 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d315f57-83b0-47de-b29c-a4cce0c34384 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:01.494580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197de925-bef6-4ab9-b6fc-2f1e193c6ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:01.495158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de925-bef6-4ab9-b6fc-2f1e193c6ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.495487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de925-bef6-4ab9-b6fc-2f1e193c6ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.495793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de925-bef6-4ab9-b6fc-2f1e193c6ed9 None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.497014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197de925-bef6-4ab9-b6fc-2f1e193c6ed9 None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:01.498505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197de925-bef6-4ab9-b6fc-2f1e193c6ed9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:01.500902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 219/442] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:53:01 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:01.506759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197de925-bef6-4ab9-b6fc-2f1e193c6ed9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:01.531318 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082731d4-c80c-4f2a-8096-7789f1bce4b8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:01.570812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e8316-1776-4d93-8497-eda43c9b6a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:01.571467 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e8316-1776-4d93-8497-eda43c9b6a77 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.571747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e8316-1776-4d93-8497-eda43c9b6a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.572023 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e8316-1776-4d93-8497-eda43c9b6a77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.586141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e8316-1776-4d93-8497-eda43c9b6a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-c86e8316-1776-4d93-8497-eda43c9b6a77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:01.593564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:01.594480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:53:01.595288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:01.597930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 224/443] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:01.599162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e8316-1776-4d93-8497-eda43c9b6a77 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:01.604032 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:01.623818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197de925-bef6-4ab9-b6fc-2f1e193c6ed9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:01.664061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ed0d2-c999-46b3-a26e-946376666e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:01.664713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ed0d2-c999-46b3-a26e-946376666e99 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.665024 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ed0d2-c999-46b3-a26e-946376666e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.665303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ed0d2-c999-46b3-a26e-946376666e99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.677820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ed0d2-c999-46b3-a26e-946376666e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ef6ed0d2-c999-46b3-a26e-946376666e99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:01.682806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:01.683714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:01.684536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 220/444] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:01.685260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ed0d2-c999-46b3-a26e-946376666e99 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:01.708323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:01.722171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e8316-1776-4d93-8497-eda43c9b6a77 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:01.771177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9029d34-5f83-4bb7-bde6-301be174ab8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:01.771725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9029d34-5f83-4bb7-bde6-301be174ab8d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.772012 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9029d34-5f83-4bb7-bde6-301be174ab8d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.772999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9029d34-5f83-4bb7-bde6-301be174ab8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.773139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9029d34-5f83-4bb7-bde6-301be174ab8d None admin] RBAC: Adding query filter params (group.id=11ba0ae8b86745989af94117b91abdd3, role.id=f4c7e5a278e34f03a86f1f88ed77db0c, scope.project.id=de970216eec1471ca4c15bf6365e4ef0) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:01.773601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9029d34-5f83-4bb7-bde6-301be174ab8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:01.775234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9029d34-5f83-4bb7-bde6-301be174ab8d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:01.782214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 225/445] 10.222.0.86 () {56 vars in 1376 bytes} [Thu Jul 28 17:53:01 2022] GET /identity/v3/role_assignments?group.id=11ba0ae8b86745989af94117b91abdd3&scope.project.id=de970216eec1471ca4c15bf6365e4ef0&role.id=f4c7e5a278e34f03a86f1f88ed77db0c => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:01.783058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9029d34-5f83-4bb7-bde6-301be174ab8d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:01.820136 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ed0d2-c999-46b3-a26e-946376666e99 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:01.862667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ef891-7fb3-4232-b443-ac15eb7969be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:01.863311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ef891-7fb3-4232-b443-ac15eb7969be None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.863882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ef891-7fb3-4232-b443-ac15eb7969be None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.863882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ef891-7fb3-4232-b443-ac15eb7969be None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.864615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ef891-7fb3-4232-b443-ac15eb7969be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:01.864926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ef891-7fb3-4232-b443-ac15eb7969be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:01.866376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ef891-7fb3-4232-b443-ac15eb7969be None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:01.869842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:01.876773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 221/446] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:01.882677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ef891-7fb3-4232-b443-ac15eb7969be None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:01.911015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9029d34-5f83-4bb7-bde6-301be174ab8d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:01.954465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ee341-69cf-49f7-b422-4f86a1c82ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:01.955112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ee341-69cf-49f7-b422-4f86a1c82ada None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:01.955403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ee341-69cf-49f7-b422-4f86a1c82ada None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:01.955682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ee341-69cf-49f7-b422-4f86a1c82ada None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:01.956409 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ee341-69cf-49f7-b422-4f86a1c82ada None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:01.956721 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ee341-69cf-49f7-b422-4f86a1c82ada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:01.958135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ee341-69cf-49f7-b422-4f86a1c82ada None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:01.967114 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 226/447] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:01.973364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ee341-69cf-49f7-b422-4f86a1c82ada None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:01.976955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:02.000029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ef891-7fb3-4232-b443-ac15eb7969be None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:02.043854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa34359-e66a-4437-acb5-85f501027ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:02.044711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa34359-e66a-4437-acb5-85f501027ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:02.045069 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa34359-e66a-4437-acb5-85f501027ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:02.045349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa34359-e66a-4437-acb5-85f501027ef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:02.092302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ee341-69cf-49f7-b422-4f86a1c82ada None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:02.128823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:02.136785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39510d-f483-4417-8f8d-6fffad49f12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:02.137448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39510d-f483-4417-8f8d-6fffad49f12c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:02.137727 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39510d-f483-4417-8f8d-6fffad49f12c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:02.137997 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39510d-f483-4417-8f8d-6fffad49f12c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:02.142620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa34359-e66a-4437-acb5-85f501027ef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:02.144086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa34359-e66a-4437-acb5-85f501027ef6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:02.223762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39510d-f483-4417-8f8d-6fffad49f12c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:02.223762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39510d-f483-4417-8f8d-6fffad49f12c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:02.245971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:02.273056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 222/448] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:53:02.275525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa34359-e66a-4437-acb5-85f501027ef6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:02.337437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 227/449] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:53:02.342764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39510d-f483-4417-8f8d-6fffad49f12c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:02.371759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:02.377954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa34359-e66a-4437-acb5-85f501027ef6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:02.413206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cad0ad-981c-435b-a3c2-17fb96207e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:02.413849 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cad0ad-981c-435b-a3c2-17fb96207e90 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:02.414132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cad0ad-981c-435b-a3c2-17fb96207e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:02.414410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cad0ad-981c-435b-a3c2-17fb96207e90 None admin] PATH_INFO: `/v3/projects/1e2fa6632a6f4745b4a2406290657577` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:02.415592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cad0ad-981c-435b-a3c2-17fb96207e90 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2fa6632a6f4745b4a2406290657577)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:02.417086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cad0ad-981c-435b-a3c2-17fb96207e90 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:02.420180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 223/450] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:53:02 2022] GET /identity/v3/projects/1e2fa6632a6f4745b4a2406290657577 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:02.424165 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cad0ad-981c-435b-a3c2-17fb96207e90 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:02.446063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39510d-f483-4417-8f8d-6fffad49f12c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:02.473835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:02.479930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99673e5f-09cc-4784-aa22-bfe6c5be78b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:02.480576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99673e5f-09cc-4784-aa22-bfe6c5be78b4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:02.485231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99673e5f-09cc-4784-aa22-bfe6c5be78b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:02.485231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99673e5f-09cc-4784-aa22-bfe6c5be78b4 None admin] PATH_INFO: `/v3/projects/e2f95f7ebd0046e996236ac2938731ae` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:02.485231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99673e5f-09cc-4784-aa22-bfe6c5be78b4 None admin] RBAC: Authorizing `identity:get_project(project_id=e2f95f7ebd0046e996236ac2938731ae)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:02.485231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99673e5f-09cc-4784-aa22-bfe6c5be78b4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:02.485566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 228/451] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:53:02 2022] GET /identity/v3/projects/e2f95f7ebd0046e996236ac2938731ae => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:02.490986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99673e5f-09cc-4784-aa22-bfe6c5be78b4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:02.525275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cad0ad-981c-435b-a3c2-17fb96207e90 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:02.558878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32eaa3-b376-4c6c-ba7f-293d26a9c78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:02.559507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32eaa3-b376-4c6c-ba7f-293d26a9c78e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:02.559796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32eaa3-b376-4c6c-ba7f-293d26a9c78e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:02.560066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32eaa3-b376-4c6c-ba7f-293d26a9c78e None admin] PATH_INFO: `/v3/groups/11ba0ae8b86745989af94117b91abdd3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:02.561226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32eaa3-b376-4c6c-ba7f-293d26a9c78e None admin] RBAC: Authorizing `identity:get_group(group_id=11ba0ae8b86745989af94117b91abdd3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:02.562649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32eaa3-b376-4c6c-ba7f-293d26a9c78e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:02.564225 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 224/452] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:53:02 2022] GET /identity/v3/groups/11ba0ae8b86745989af94117b91abdd3 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:02.569936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32eaa3-b376-4c6c-ba7f-293d26a9c78e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:02.586248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:02.589879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99673e5f-09cc-4784-aa22-bfe6c5be78b4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:02.622881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe770b-c9d5-4de5-86f3-ca39561c975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:02.623530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe770b-c9d5-4de5-86f3-ca39561c975b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:02.623816 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe770b-c9d5-4de5-86f3-ca39561c975b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:02.624091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe770b-c9d5-4de5-86f3-ca39561c975b None admin] PATH_INFO: `/v3/groups/1217606c342f417589bb5ab9445c5504` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:02.625285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe770b-c9d5-4de5-86f3-ca39561c975b None admin] RBAC: Authorizing `identity:get_group(group_id=1217606c342f417589bb5ab9445c5504)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:02.626972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe770b-c9d5-4de5-86f3-ca39561c975b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:02.628565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 229/453] 10.222.0.86 () {56 vars in 1180 bytes} [Thu Jul 28 17:53:02 2022] GET /identity/v3/groups/1217606c342f417589bb5ab9445c5504 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:02.634272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe770b-c9d5-4de5-86f3-ca39561c975b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:02.670467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32eaa3-b376-4c6c-ba7f-293d26a9c78e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:02.686525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:02.704649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0789bb9-5ad6-4f36-a80e-e7e724ed792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:02.705293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0789bb9-5ad6-4f36-a80e-e7e724ed792d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:02.705597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0789bb9-5ad6-4f36-a80e-e7e724ed792d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:02.705854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0789bb9-5ad6-4f36-a80e-e7e724ed792d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:02.706649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0789bb9-5ad6-4f36-a80e-e7e724ed792d None admin] RBAC: Adding query filter params (group.id=11ba0ae8b86745989af94117b91abdd3, role.id=f4c7e5a278e34f03a86f1f88ed77db0c, scope.project.id=1e2fa6632a6f4745b4a2406290657577) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:02.706960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0789bb9-5ad6-4f36-a80e-e7e724ed792d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:02.708390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0789bb9-5ad6-4f36-a80e-e7e724ed792d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:02.714266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 225/454] 10.222.0.86 () {56 vars in 1376 bytes} [Thu Jul 28 17:53:02 2022] GET /identity/v3/role_assignments?group.id=11ba0ae8b86745989af94117b91abdd3&scope.project.id=1e2fa6632a6f4745b4a2406290657577&role.id=f4c7e5a278e34f03a86f1f88ed77db0c => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:02.733244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe770b-c9d5-4de5-86f3-ca39561c975b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:02.771247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95da3d-6976-46a2-b148-9907031aebb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:02.771888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95da3d-6976-46a2-b148-9907031aebb4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:02.772268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95da3d-6976-46a2-b148-9907031aebb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:02.772550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95da3d-6976-46a2-b148-9907031aebb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:02.773397 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95da3d-6976-46a2-b148-9907031aebb4 None admin] RBAC: Adding query filter params (group.id=1217606c342f417589bb5ab9445c5504, role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=e2f95f7ebd0046e996236ac2938731ae) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:02.773714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95da3d-6976-46a2-b148-9907031aebb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:02.775165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95da3d-6976-46a2-b148-9907031aebb4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:02.781895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 230/455] 10.222.0.86 () {56 vars in 1376 bytes} [Thu Jul 28 17:53:02 2022] GET /identity/v3/role_assignments?group.id=1217606c342f417589bb5ab9445c5504&scope.project.id=e2f95f7ebd0046e996236ac2938731ae&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:05.234607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28ec0e-ec81-4a44-9308-1cc034301fb5 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:05.235139 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28ec0e-ec81-4a44-9308-1cc034301fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:05.235444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28ec0e-ec81-4a44-9308-1cc034301fb5 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:05.237234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 226/456] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:05.250854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523aad8-8776-4aec-ad87-28b29a41e16b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:05.251567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523aad8-8776-4aec-ad87-28b29a41e16b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:05.251651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523aad8-8776-4aec-ad87-28b29a41e16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:05.277042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d523aad8-8776-4aec-ad87-28b29a41e16b None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:05.278281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d523aad8-8776-4aec-ad87-28b29a41e16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:05.306602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:05.363507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 231/457] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:05.399278 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28ec0e-ec81-4a44-9308-1cc034301fb5 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:05.484252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82297e-5155-48e9-bfd6-dcb42ef059c1 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:05.484573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82297e-5155-48e9-bfd6-dcb42ef059c1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:05.484892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82297e-5155-48e9-bfd6-dcb42ef059c1 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:05.486440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 232/458] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:53:05.491487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd47932-00d0-4230-9461-529eadbc7a0e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:05.491781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd47932-00d0-4230-9461-529eadbc7a0e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:05.492062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd47932-00d0-4230-9461-529eadbc7a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:05.524803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28ec0e-ec81-4a44-9308-1cc034301fb5 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:05.532152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-afd47932-00d0-4230-9461-529eadbc7a0e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:05.533405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-afd47932-00d0-4230-9461-529eadbc7a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:05.561718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c065fe-111e-4e1c-a1b8-0dabd8711063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:05.562411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c065fe-111e-4e1c-a1b8-0dabd8711063 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:05.562697 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c065fe-111e-4e1c-a1b8-0dabd8711063 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:05.562976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c065fe-111e-4e1c-a1b8-0dabd8711063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:05.574566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c065fe-111e-4e1c-a1b8-0dabd8711063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:05.576149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c065fe-111e-4e1c-a1b8-0dabd8711063 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a0c065fe-111e-4e1c-a1b8-0dabd8711063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:05.592997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:05.594545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:05.595575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:05.595575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:05.595575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:05.595575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:05.595575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 227/459] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:05.600353 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c065fe-111e-4e1c-a1b8-0dabd8711063 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:05.637115 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:05.691540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 233/460] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:53:05.709221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c065fe-111e-4e1c-a1b8-0dabd8711063 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:05.727953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd47932-00d0-4230-9461-529eadbc7a0e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:05.761821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f874c4-df14-47a4-85c6-03e3f8040482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:05.761821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f874c4-df14-47a4-85c6-03e3f8040482 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:05.761821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f874c4-df14-47a4-85c6-03e3f8040482 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:05.761821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f874c4-df14-47a4-85c6-03e3f8040482 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:05.762322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f874c4-df14-47a4-85c6-03e3f8040482 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:05.762322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f874c4-df14-47a4-85c6-03e3f8040482 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:05.762322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f874c4-df14-47a4-85c6-03e3f8040482 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:05.772563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 228/461] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:05.785846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f874c4-df14-47a4-85c6-03e3f8040482 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:05.833571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:05.844753 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd47932-00d0-4230-9461-529eadbc7a0e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:05.911817 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6330fa8-dc95-4e7d-88d4-115104018eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:05.912482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6330fa8-dc95-4e7d-88d4-115104018eef None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:05.912765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6330fa8-dc95-4e7d-88d4-115104018eef None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:05.913068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6330fa8-dc95-4e7d-88d4-115104018eef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:05.935284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6330fa8-dc95-4e7d-88d4-115104018eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:05.937271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6330fa8-dc95-4e7d-88d4-115104018eef None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:05.960015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f6330fa8-dc95-4e7d-88d4-115104018eef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:05.965606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:05.966511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:05.967428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:05.967428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:05.967428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:05.967428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:05.969860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 234/462] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:05.978536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6330fa8-dc95-4e7d-88d4-115104018eef None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:05.981056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f874c4-df14-47a4-85c6-03e3f8040482 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:06.024770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1cd89-ad6b-46b2-9344-91aea05bff5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:06.025477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1cd89-ad6b-46b2-9344-91aea05bff5c None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.025776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1cd89-ad6b-46b2-9344-91aea05bff5c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.026056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1cd89-ad6b-46b2-9344-91aea05bff5c None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.026749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1cd89-ad6b-46b2-9344-91aea05bff5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:06.029412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1cd89-ad6b-46b2-9344-91aea05bff5c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:06.121687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6330fa8-dc95-4e7d-88d4-115104018eef None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:06.141344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 229/463] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:53:05 2022] POST /identity/v3/users => generated 246 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:53:06.143243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabf85b-170f-4c54-8a31-8b38c988fe7f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.143534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabf85b-170f-4c54-8a31-8b38c988fe7f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.143816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabf85b-170f-4c54-8a31-8b38c988fe7f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.149736 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 230/464] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:06.153490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad17471-ed42-494c-8bac-9a556fad4f7d None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.157120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad17471-ed42-494c-8bac-9a556fad4f7d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.163315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad17471-ed42-494c-8bac-9a556fad4f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.178586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1ad17471-ed42-494c-8bac-9a556fad4f7d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:06.179468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e83c81-c523-4be8-816c-6248045b68cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:06.180022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1ad17471-ed42-494c-8bac-9a556fad4f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:06.180600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e83c81-c523-4be8-816c-6248045b68cd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.181117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e83c81-c523-4be8-816c-6248045b68cd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.181591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e83c81-c523-4be8-816c-6248045b68cd None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.182648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e83c81-c523-4be8-816c-6248045b68cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:06.183031 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e83c81-c523-4be8-816c-6248045b68cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:06.184828 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e83c81-c523-4be8-816c-6248045b68cd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:06.203755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 235/465] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:06.203755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225fb72-3ea2-4829-8cd7-07b32eeab153 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.203755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225fb72-3ea2-4829-8cd7-07b32eeab153 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.203755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225fb72-3ea2-4829-8cd7-07b32eeab153 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.203755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 236/466] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:06.210150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225fb72-3ea2-4829-8cd7-07b32eeab153 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:06.216364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:06.295967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 231/467] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:53:06.304019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff58d4-6067-4367-a67b-15995784fbeb None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.304019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff58d4-6067-4367-a67b-15995784fbeb None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.304019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff58d4-6067-4367-a67b-15995784fbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.328935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d0ff58d4-6067-4367-a67b-15995784fbeb None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:06.330174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d0ff58d4-6067-4367-a67b-15995784fbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:06.360135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225fb72-3ea2-4829-8cd7-07b32eeab153 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:06.407167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4f385-e2bd-48c8-a997-bda90e9d20d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:06.407936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4f385-e2bd-48c8-a997-bda90e9d20d0 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.408290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4f385-e2bd-48c8-a997-bda90e9d20d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.408646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4f385-e2bd-48c8-a997-bda90e9d20d0 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.409447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4f385-e2bd-48c8-a997-bda90e9d20d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:06.410957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4f385-e2bd-48c8-a997-bda90e9d20d0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:06.426992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 232/468] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:06.426992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff58d4-6067-4367-a67b-15995784fbeb None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:06.436708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:06.525981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 237/469] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:53:06 2022] POST /identity/v3/users => generated 248 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:06.526984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4f385-e2bd-48c8-a997-bda90e9d20d0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:06.535355 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff58d4-6067-4367-a67b-15995784fbeb None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:06.586590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63c792-d739-41d2-a1d7-956f925f1aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:06.587202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63c792-d739-41d2-a1d7-956f925f1aff None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.587489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63c792-d739-41d2-a1d7-956f925f1aff None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.587768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63c792-d739-41d2-a1d7-956f925f1aff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.617735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63c792-d739-41d2-a1d7-956f925f1aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:06.619272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63c792-d739-41d2-a1d7-956f925f1aff None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ed63c792-d739-41d2-a1d7-956f925f1aff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:06.639309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:06.640921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:06.641900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:06.641900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:06.641900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:06.641900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:06.641900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 233/470] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:06.642691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67672-3387-4a6f-a29a-bfe83277cfc4 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.643181 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67672-3387-4a6f-a29a-bfe83277cfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.643465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67672-3387-4a6f-a29a-bfe83277cfc4 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.644777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 234/471] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:06.654595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf67672-3387-4a6f-a29a-bfe83277cfc4 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:06.654727 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:06.707429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4f385-e2bd-48c8-a997-bda90e9d20d0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:06.722874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:06.762426 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e36c5-2b03-46ed-9c12-c1196525b7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:06.764151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e36c5-2b03-46ed-9c12-c1196525b7ca None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.764151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e36c5-2b03-46ed-9c12-c1196525b7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.764151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e36c5-2b03-46ed-9c12-c1196525b7ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.775465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf67672-3387-4a6f-a29a-bfe83277cfc4 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:06.776477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e36c5-2b03-46ed-9c12-c1196525b7ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:06.778179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e36c5-2b03-46ed-9c12-c1196525b7ca None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-321e36c5-2b03-46ed-9c12-c1196525b7ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:06.796242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:06.797842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:06.798731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:06.798731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:06.798731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:06.798731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:06.800428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 238/472] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:06.801852 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816f62c-bd1f-40b6-a723-0a213aeddc8f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.802304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816f62c-bd1f-40b6-a723-0a213aeddc8f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.802808 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816f62c-bd1f-40b6-a723-0a213aeddc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.822566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dbbc5-6dfa-45e9-82d6-2628aad3212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:06.822869 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dbbc5-6dfa-45e9-82d6-2628aad3212e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:06.823291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dbbc5-6dfa-45e9-82d6-2628aad3212e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:06.823645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dbbc5-6dfa-45e9-82d6-2628aad3212e None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:06.824413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6816f62c-bd1f-40b6-a723-0a213aeddc8f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:06.824714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dbbc5-6dfa-45e9-82d6-2628aad3212e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:06.825171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dbbc5-6dfa-45e9-82d6-2628aad3212e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:06.826279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6816f62c-bd1f-40b6-a723-0a213aeddc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:06.826825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dbbc5-6dfa-45e9-82d6-2628aad3212e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:06.844160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 235/473] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:06.845022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dbbc5-6dfa-45e9-82d6-2628aad3212e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:06.910311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 239/474] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:06.911213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816f62c-bd1f-40b6-a723-0a213aeddc8f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:06.979326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:06.982810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dbbc5-6dfa-45e9-82d6-2628aad3212e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:07.015360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:07.029820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cff4a1-09a4-4ea3-94df-c0115f108016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:07.030674 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cff4a1-09a4-4ea3-94df-c0115f108016 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:07.031021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cff4a1-09a4-4ea3-94df-c0115f108016 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:07.031304 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cff4a1-09a4-4ea3-94df-c0115f108016 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:07.032014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cff4a1-09a4-4ea3-94df-c0115f108016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:07.032320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cff4a1-09a4-4ea3-94df-c0115f108016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:07.033761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cff4a1-09a4-4ea3-94df-c0115f108016 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:07.046690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 236/475] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:07.047742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cff4a1-09a4-4ea3-94df-c0115f108016 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:07.053893 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816f62c-bd1f-40b6-a723-0a213aeddc8f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:07.091932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d305f8-279e-4a1c-b115-0ddf7c67bad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:07.092842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d305f8-279e-4a1c-b115-0ddf7c67bad3 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:07.093283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d305f8-279e-4a1c-b115-0ddf7c67bad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:07.093637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d305f8-279e-4a1c-b115-0ddf7c67bad3 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:07.094515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d305f8-279e-4a1c-b115-0ddf7c67bad3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:07.096009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d305f8-279e-4a1c-b115-0ddf7c67bad3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:07.184910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cff4a1-09a4-4ea3-94df-c0115f108016 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:07.225728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c030e92-fa22-4775-b3fc-e7becd000fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:07.226903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c030e92-fa22-4775-b3fc-e7becd000fcb None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:07.227405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c030e92-fa22-4775-b3fc-e7becd000fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:07.227910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c030e92-fa22-4775-b3fc-e7becd000fcb None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:07.229011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c030e92-fa22-4775-b3fc-e7becd000fcb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:07.231037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c030e92-fa22-4775-b3fc-e7becd000fcb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:07.233516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 240/476] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:53:06 2022] POST /identity/v3/users => generated 248 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:07.235026 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d305f8-279e-4a1c-b115-0ddf7c67bad3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:07.305482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.sql.core [None req-8c030e92-fa22-4775-b3fc-e7becd000fcb None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 17:53:07.305482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 17:53:07.305482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [parameters: {'id': '5715791e551c4a1f81e662e05b5f4485', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 17:53:07.305482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80464) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8c030e92-fa22-4775-b3fc-e7becd000fcb None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result.read() Jul 28 17:53:07.336234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:53:07.337227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application yield resource Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise self.value Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application session.commit() Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 17:53:07.338201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.session.flush() Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application compat.raise_( Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 17:53:07.339299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:53:07.340217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result.read() Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:53:07.342298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [parameters: {'id': '5715791e551c4a1f81e662e05b5f4485', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:07.344937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 17:53:07.347011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:07.350640 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 237/477] 10.222.0.86 () {60 vars in 1131 bytes} [Thu Jul 28 17:53:07 2022] POST /identity/v3/roles => generated 144 bytes in 303 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:07.356352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c030e92-fa22-4775-b3fc-e7becd000fcb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:07.430386 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:07.438371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d305f8-279e-4a1c-b115-0ddf7c67bad3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:07.446406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:07.482248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ddf00-337c-4c6a-aec2-e72876d8b369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:07.486182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ddf00-337c-4c6a-aec2-e72876d8b369 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:07.486182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ddf00-337c-4c6a-aec2-e72876d8b369 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:07.486182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ddf00-337c-4c6a-aec2-e72876d8b369 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:07.486182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ddf00-337c-4c6a-aec2-e72876d8b369 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:07.487730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ddf00-337c-4c6a-aec2-e72876d8b369 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:07.516563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c030e92-fa22-4775-b3fc-e7becd000fcb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:07.557052 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906d156-a691-48ad-a6a5-35b8aff57032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:07.557714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906d156-a691-48ad-a6a5-35b8aff57032 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:07.558002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906d156-a691-48ad-a6a5-35b8aff57032 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:07.558285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906d156-a691-48ad-a6a5-35b8aff57032 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:07.569422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 241/478] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:53:07 2022] POST /identity/v3/users => generated 249 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:07.579206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906d156-a691-48ad-a6a5-35b8aff57032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c906d156-a691-48ad-a6a5-35b8aff57032 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:07.585106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:07.586113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:53:07.586940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:07.587636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 238/479] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:07.594558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ddf00-337c-4c6a-aec2-e72876d8b369 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:07.714764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ddf00-337c-4c6a-aec2-e72876d8b369 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:07.752863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ef787-eb4f-4ec0-be97-9e33fe3f1b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:07.753758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ef787-eb4f-4ec0-be97-9e33fe3f1b5d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:07.754215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ef787-eb4f-4ec0-be97-9e33fe3f1b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:07.754663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ef787-eb4f-4ec0-be97-9e33fe3f1b5d None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:07.755543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ef787-eb4f-4ec0-be97-9e33fe3f1b5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:07.756029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ef787-eb4f-4ec0-be97-9e33fe3f1b5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:07.757707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ef787-eb4f-4ec0-be97-9e33fe3f1b5d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:07.767495 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 242/480] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:08.403269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267ae7d-f2ba-434a-b0a8-81b71cbe60c7 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:08.403819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267ae7d-f2ba-434a-b0a8-81b71cbe60c7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:08.403819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267ae7d-f2ba-434a-b0a8-81b71cbe60c7 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:08.405200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 239/481] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:08.411076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6540507-cc33-4c9d-80fc-b40443f84466 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:08.411368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6540507-cc33-4c9d-80fc-b40443f84466 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:08.411649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6540507-cc33-4c9d-80fc-b40443f84466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:08.449148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a6540507-cc33-4c9d-80fc-b40443f84466 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:08.451145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a6540507-cc33-4c9d-80fc-b40443f84466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:08.465751 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:08.564838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 243/482] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:08.600472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267ae7d-f2ba-434a-b0a8-81b71cbe60c7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:08.733673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267ae7d-f2ba-434a-b0a8-81b71cbe60c7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:08.735959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:08.770042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56d166-9c56-49c1-9738-19fb962efbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:08.770688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56d166-9c56-49c1-9738-19fb962efbda None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:08.770972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56d166-9c56-49c1-9738-19fb962efbda None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:08.771246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56d166-9c56-49c1-9738-19fb962efbda None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:08.781812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56d166-9c56-49c1-9738-19fb962efbda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8b56d166-9c56-49c1-9738-19fb962efbda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:08.786848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:08.787751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:08.788569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 240/483] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:08.794787 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6540507-cc33-4c9d-80fc-b40443f84466 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:08.917771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6540507-cc33-4c9d-80fc-b40443f84466 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:08.955621 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fb65d-ad6c-4b85-bd81-52376a01c84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:08.956244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb65d-ad6c-4b85-bd81-52376a01c84e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:08.956524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb65d-ad6c-4b85-bd81-52376a01c84e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:08.956799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb65d-ad6c-4b85-bd81-52376a01c84e None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:08.957547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fb65d-ad6c-4b85-bd81-52376a01c84e None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:08.957848 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fb65d-ad6c-4b85-bd81-52376a01c84e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:08.959208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fb65d-ad6c-4b85-bd81-52376a01c84e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:08.967371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 244/484] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:08.975056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56d166-9c56-49c1-9738-19fb962efbda None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:09.083907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56d166-9c56-49c1-9738-19fb962efbda None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:09.122702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67044901-52f2-45a4-aee5-10144518e9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:09.123337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67044901-52f2-45a4-aee5-10144518e9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:09.123613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67044901-52f2-45a4-aee5-10144518e9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:09.123885 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67044901-52f2-45a4-aee5-10144518e9e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:09.135345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67044901-52f2-45a4-aee5-10144518e9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:09.136873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67044901-52f2-45a4-aee5-10144518e9e2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-67044901-52f2-45a4-aee5-10144518e9e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:09.151873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:09.152758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:09.153670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:09.153670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:09.153670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:09.153670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:09.153670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 241/485] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:09.158872 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fb65d-ad6c-4b85-bd81-52376a01c84e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:09.177360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:09.224059 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4ec79-3e59-4022-9c5c-f886540147d8 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:09.224383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4ec79-3e59-4022-9c5c-f886540147d8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:09.224656 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4ec79-3e59-4022-9c5c-f886540147d8 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:09.226034 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 242/486] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:53:09.238944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16252a72-f4b1-4d6d-9004-d8a20d6dff0d None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:09.239260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16252a72-f4b1-4d6d-9004-d8a20d6dff0d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:09.239509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16252a72-f4b1-4d6d-9004-d8a20d6dff0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:09.268538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-16252a72-f4b1-4d6d-9004-d8a20d6dff0d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:09.269774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-16252a72-f4b1-4d6d-9004-d8a20d6dff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:09.273794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fb65d-ad6c-4b85-bd81-52376a01c84e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:09.294940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:09.313276 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ee7a9-d75d-43d9-8468-cab9c23265d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:09.313914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ee7a9-d75d-43d9-8468-cab9c23265d2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:09.314198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ee7a9-d75d-43d9-8468-cab9c23265d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:09.314473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ee7a9-d75d-43d9-8468-cab9c23265d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:09.315182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ee7a9-d75d-43d9-8468-cab9c23265d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:09.315519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ee7a9-d75d-43d9-8468-cab9c23265d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:09.317140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ee7a9-d75d-43d9-8468-cab9c23265d2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:09.331117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 245/487] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:09.337288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ee7a9-d75d-43d9-8468-cab9c23265d2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:09.423211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 243/488] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:09.444056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ee7a9-d75d-43d9-8468-cab9c23265d2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:09.456657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:09.457551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16252a72-f4b1-4d6d-9004-d8a20d6dff0d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:09.484875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18761e8-7e82-4f80-a6d4-9e8ca186a58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:09.485557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18761e8-7e82-4f80-a6d4-9e8ca186a58d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:09.485848 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18761e8-7e82-4f80-a6d4-9e8ca186a58d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:09.486128 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18761e8-7e82-4f80-a6d4-9e8ca186a58d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:09.499538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18761e8-7e82-4f80-a6d4-9e8ca186a58d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:09.502249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18761e8-7e82-4f80-a6d4-9e8ca186a58d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f18761e8-7e82-4f80-a6d4-9e8ca186a58d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:09.520794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:09.521761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:09.522673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:09.522673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:09.522673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:09.522673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:09.522673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:09.522673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:09.522673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:09.522673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:09.522673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:09.522673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:09.533251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 246/489] 10.222.0.86 () {56 vars in 1093 bytes} [Thu Jul 28 17:53:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:09.534101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18761e8-7e82-4f80-a6d4-9e8ca186a58d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:09.628441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16252a72-f4b1-4d6d-9004-d8a20d6dff0d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:09.685165 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17f15c-c8d3-4853-8ad0-ff1b9af677f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:09.685817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17f15c-c8d3-4853-8ad0-ff1b9af677f6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:09.686099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17f15c-c8d3-4853-8ad0-ff1b9af677f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:09.686370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17f15c-c8d3-4853-8ad0-ff1b9af677f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:09.708317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17f15c-c8d3-4853-8ad0-ff1b9af677f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-be17f15c-c8d3-4853-8ad0-ff1b9af677f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:09.717700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:09.719151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:09.719967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:09.723374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 244/490] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:09.726626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18761e8-7e82-4f80-a6d4-9e8ca186a58d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:09.729243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17f15c-c8d3-4853-8ad0-ff1b9af677f6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:09.741591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:09.759366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:09.773771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121ecbc-d864-4059-96d1-0d034f0ca6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:09.774537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121ecbc-d864-4059-96d1-0d034f0ca6ce None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:09.774819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121ecbc-d864-4059-96d1-0d034f0ca6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:09.775092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121ecbc-d864-4059-96d1-0d034f0ca6ce None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:09.775958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121ecbc-d864-4059-96d1-0d034f0ca6ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:09.776268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121ecbc-d864-4059-96d1-0d034f0ca6ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:09.778522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121ecbc-d864-4059-96d1-0d034f0ca6ce None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:09.855098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 247/491] 10.222.0.86 () {56 vars in 1133 bytes} [Thu Jul 28 17:53:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:09.867781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121ecbc-d864-4059-96d1-0d034f0ca6ce None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:09.898670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17f15c-c8d3-4853-8ad0-ff1b9af677f6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:09.950332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ef8d9-5652-4d57-aa3b-cd5b1f1969ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:09.951037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ef8d9-5652-4d57-aa3b-cd5b1f1969ef None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:09.951318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ef8d9-5652-4d57-aa3b-cd5b1f1969ef None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:09.951645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ef8d9-5652-4d57-aa3b-cd5b1f1969ef None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:09.952443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ef8d9-5652-4d57-aa3b-cd5b1f1969ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:09.952800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ef8d9-5652-4d57-aa3b-cd5b1f1969ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:09.954215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ef8d9-5652-4d57-aa3b-cd5b1f1969ef None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:09.971573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 245/492] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:09.978460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ef8d9-5652-4d57-aa3b-cd5b1f1969ef None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:09.986162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121ecbc-d864-4059-96d1-0d034f0ca6ce None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:10.022868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b0523-7df0-43f4-aa1c-dbc2ca612b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:10.023503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b0523-7df0-43f4-aa1c-dbc2ca612b0a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.023790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b0523-7df0-43f4-aa1c-dbc2ca612b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.024074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b0523-7df0-43f4-aa1c-dbc2ca612b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.060151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:10.085125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:10.094874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ef8d9-5652-4d57-aa3b-cd5b1f1969ef None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:10.099859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b0523-7df0-43f4-aa1c-dbc2ca612b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:10.101539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b0523-7df0-43f4-aa1c-dbc2ca612b0a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:10.179096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54a3fc-5aa0-45aa-bc56-121d4345b497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:10.180708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54a3fc-5aa0-45aa-bc56-121d4345b497 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.181157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54a3fc-5aa0-45aa-bc56-121d4345b497 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.181540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54a3fc-5aa0-45aa-bc56-121d4345b497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.195760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54a3fc-5aa0-45aa-bc56-121d4345b497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:10.197964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54a3fc-5aa0-45aa-bc56-121d4345b497 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-5e54a3fc-5aa0-45aa-bc56-121d4345b497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:10.217370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:10.218969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:10.219833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:10.219833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:10.219833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:10.219833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:10.220733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 246/493] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:10.225009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9aa6e-a209-4850-8e97-06ece1e109b7 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.225009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9aa6e-a209-4850-8e97-06ece1e109b7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.225009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9aa6e-a209-4850-8e97-06ece1e109b7 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.228561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 247/494] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:10.229516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9aa6e-a209-4850-8e97-06ece1e109b7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:10.271626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 248/495] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:10.273007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c816343-6475-40e2-9a3c-ffeccce03e34 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.273304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c816343-6475-40e2-9a3c-ffeccce03e34 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.273613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c816343-6475-40e2-9a3c-ffeccce03e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.300276 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3c816343-6475-40e2-9a3c-ffeccce03e34 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:10.301700 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3c816343-6475-40e2-9a3c-ffeccce03e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:10.359890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9aa6e-a209-4850-8e97-06ece1e109b7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:10.373055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 249/496] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:10.373984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c816343-6475-40e2-9a3c-ffeccce03e34 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:10.380948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:10.389356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:10.395312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abacb8-67d2-4e2e-96fa-8f3952cc54c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:10.395936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abacb8-67d2-4e2e-96fa-8f3952cc54c9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.396302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abacb8-67d2-4e2e-96fa-8f3952cc54c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.396586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abacb8-67d2-4e2e-96fa-8f3952cc54c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.397336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abacb8-67d2-4e2e-96fa-8f3952cc54c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:10.397641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abacb8-67d2-4e2e-96fa-8f3952cc54c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:10.399010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abacb8-67d2-4e2e-96fa-8f3952cc54c9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:10.411184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 248/497] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:10.412411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138ca61-3a26-4b1c-ae42-1854d3b31c04 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.412723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138ca61-3a26-4b1c-ae42-1854d3b31c04 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.412999 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138ca61-3a26-4b1c-ae42-1854d3b31c04 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.414499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 249/498] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:10.415677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313927-674d-45f2-b038-bda9a6a62f11 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.416268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313927-674d-45f2-b038-bda9a6a62f11 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.417093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313927-674d-45f2-b038-bda9a6a62f11 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.418855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 250/499] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:10.419761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18313927-674d-45f2-b038-bda9a6a62f11 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:10.479547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c816343-6475-40e2-9a3c-ffeccce03e34 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:10.513642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c5337-f075-4d3c-b52b-02b25dcf2b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:10.514284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c5337-f075-4d3c-b52b-02b25dcf2b48 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.514562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c5337-f075-4d3c-b52b-02b25dcf2b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.514836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c5337-f075-4d3c-b52b-02b25dcf2b48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.524515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c5337-f075-4d3c-b52b-02b25dcf2b48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:10.525909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18313927-674d-45f2-b038-bda9a6a62f11 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:10.526358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c5337-f075-4d3c-b52b-02b25dcf2b48 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-964c5337-f075-4d3c-b52b-02b25dcf2b48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:10.540892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:10.542020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:10.542935 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:10.542935 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:10.542935 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:10.542935 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:10.542935 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 250/500] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:10.542935 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c5337-f075-4d3c-b52b-02b25dcf2b48 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:10.567305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43154a1a-ef03-4e2a-b554-65d8742cc825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:10.567953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43154a1a-ef03-4e2a-b554-65d8742cc825 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.568239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43154a1a-ef03-4e2a-b554-65d8742cc825 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.568524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43154a1a-ef03-4e2a-b554-65d8742cc825 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.579787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43154a1a-ef03-4e2a-b554-65d8742cc825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-43154a1a-ef03-4e2a-b554-65d8742cc825 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:10.584716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:10.585603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:10.586414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 251/501] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:10.587342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa4155-7384-4411-9c98-c91be3bad181 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.587342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa4155-7384-4411-9c98-c91be3bad181 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.587717 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa4155-7384-4411-9c98-c91be3bad181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.605365 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6cfa4155-7384-4411-9c98-c91be3bad181 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:10.606567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6cfa4155-7384-4411-9c98-c91be3bad181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:10.613240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:10.641449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:10.648349 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c5337-f075-4d3c-b52b-02b25dcf2b48 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:10.678634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 252/502] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:10.679884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818b186-89fe-4128-bdf1-893b00d96696 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.680175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818b186-89fe-4128-bdf1-893b00d96696 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.680453 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818b186-89fe-4128-bdf1-893b00d96696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.691987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce78f1-057d-422a-9857-ab97fc9de434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:10.692638 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce78f1-057d-422a-9857-ab97fc9de434 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.692919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce78f1-057d-422a-9857-ab97fc9de434 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.693228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce78f1-057d-422a-9857-ab97fc9de434 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.701218 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce78f1-057d-422a-9857-ab97fc9de434 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:10.703194 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce78f1-057d-422a-9857-ab97fc9de434 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:10.705545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8818b186-89fe-4128-bdf1-893b00d96696 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:10.706758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8818b186-89fe-4128-bdf1-893b00d96696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ffce78f1-057d-422a-9857-ab97fc9de434 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:10.714628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:10.715494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:10.716368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:10.716368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:10.716368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:10.716368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:10.716368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:10.716368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:10.716368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:10.716368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:10.716368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:53:10.716368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:10.716368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 251/503] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:10.716928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce78f1-057d-422a-9857-ab97fc9de434 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:10.785068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 253/504] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:10.785068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818b186-89fe-4128-bdf1-893b00d96696 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:10.826155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce78f1-057d-422a-9857-ab97fc9de434 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:10.851767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:10.860674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83437e16-31a1-4671-9115-a50d16ff1dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:10.861420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83437e16-31a1-4671-9115-a50d16ff1dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.861770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83437e16-31a1-4671-9115-a50d16ff1dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.862055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83437e16-31a1-4671-9115-a50d16ff1dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.862756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83437e16-31a1-4671-9115-a50d16ff1dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:10.863064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83437e16-31a1-4671-9115-a50d16ff1dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:10.864445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83437e16-31a1-4671-9115-a50d16ff1dc3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:10.872511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:10.875308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818b186-89fe-4128-bdf1-893b00d96696 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:10.876480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 252/505] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:10.877343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83437e16-31a1-4671-9115-a50d16ff1dc3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:10.910470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343b895-35da-4ec2-987a-53369ea393be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:10.911102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343b895-35da-4ec2-987a-53369ea393be None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:10.911385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343b895-35da-4ec2-987a-53369ea393be None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:10.911667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343b895-35da-4ec2-987a-53369ea393be None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:10.912394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343b895-35da-4ec2-987a-53369ea393be None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:10.912702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343b895-35da-4ec2-987a-53369ea393be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:10.914107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343b895-35da-4ec2-987a-53369ea393be None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:10.921939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 254/506] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:10.922714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343b895-35da-4ec2-987a-53369ea393be None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:10.978291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83437e16-31a1-4671-9115-a50d16ff1dc3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.012769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84608a-5356-4f95-ae88-4a6e02d37b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.013444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84608a-5356-4f95-ae88-4a6e02d37b2f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.013730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84608a-5356-4f95-ae88-4a6e02d37b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.014012 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84608a-5356-4f95-ae88-4a6e02d37b2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.021875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343b895-35da-4ec2-987a-53369ea393be None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.024449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84608a-5356-4f95-ae88-4a6e02d37b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-6d84608a-5356-4f95-ae88-4a6e02d37b2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:11.029346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:11.030423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 253/507] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:11.036438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84608a-5356-4f95-ae88-4a6e02d37b2f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.055478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c334a8c-f8b4-4391-b201-118fb0ffbdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.056187 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c334a8c-f8b4-4391-b201-118fb0ffbdfa None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.056470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c334a8c-f8b4-4391-b201-118fb0ffbdfa None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.056749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c334a8c-f8b4-4391-b201-118fb0ffbdfa None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.060989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c334a8c-f8b4-4391-b201-118fb0ffbdfa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:11.060989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c334a8c-f8b4-4391-b201-118fb0ffbdfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.060989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c334a8c-f8b4-4391-b201-118fb0ffbdfa None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:11.078136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:11.130505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84608a-5356-4f95-ae88-4a6e02d37b2f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.141916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 255/508] 10.222.0.86 () {56 vars in 1137 bytes} [Thu Jul 28 17:53:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:11.142766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c334a8c-f8b4-4391-b201-118fb0ffbdfa None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.163695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2832e-a9d5-4dd6-a365-c1b903945e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.164329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2832e-a9d5-4dd6-a365-c1b903945e87 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.164606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2832e-a9d5-4dd6-a365-c1b903945e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.164876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2832e-a9d5-4dd6-a365-c1b903945e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.165570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:11.175741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2832e-a9d5-4dd6-a365-c1b903945e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.177248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2832e-a9d5-4dd6-a365-c1b903945e87 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-88b2832e-a9d5-4dd6-a365-c1b903945e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:11.191825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:11.192706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:11.193597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:11.193597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:11.193597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:11.193597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:11.193597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 254/509] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:11.193907 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2832e-a9d5-4dd6-a365-c1b903945e87 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.247038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c334a8c-f8b4-4391-b201-118fb0ffbdfa None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.280731 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee952a-e21a-4aa3-b3c7-4f64fdf43669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.281398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee952a-e21a-4aa3-b3c7-4f64fdf43669 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.281676 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee952a-e21a-4aa3-b3c7-4f64fdf43669 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.281945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee952a-e21a-4aa3-b3c7-4f64fdf43669 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.288499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee952a-e21a-4aa3-b3c7-4f64fdf43669 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.289917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee952a-e21a-4aa3-b3c7-4f64fdf43669 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:11.290835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:11.294595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2832e-a9d5-4dd6-a365-c1b903945e87 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-f2ee952a-e21a-4aa3-b3c7-4f64fdf43669 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:11.302561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:11.303410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:11.304253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:11.304253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:11.304253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:11.304253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:11.304253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:11.304253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:11.304253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:11.304253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:11.304253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 256/510] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:11.304724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee952a-e21a-4aa3-b3c7-4f64fdf43669 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.330866 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44677ac2-74a0-417a-a494-4d66ba33511c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.331596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44677ac2-74a0-417a-a494-4d66ba33511c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.331879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44677ac2-74a0-417a-a494-4d66ba33511c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.332159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44677ac2-74a0-417a-a494-4d66ba33511c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.343585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44677ac2-74a0-417a-a494-4d66ba33511c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.345064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44677ac2-74a0-417a-a494-4d66ba33511c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-44677ac2-74a0-417a-a494-4d66ba33511c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:11.359113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:11.360113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:11.361065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:11.361065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:11.361065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:11.361065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:11.361065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 255/511] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:11.361772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44677ac2-74a0-417a-a494-4d66ba33511c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.387064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:11.408302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee952a-e21a-4aa3-b3c7-4f64fdf43669 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.444917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed56dec-6641-4a6f-bd30-748c69e115f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.445570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed56dec-6641-4a6f-bd30-748c69e115f2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.445852 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed56dec-6641-4a6f-bd30-748c69e115f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.446130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed56dec-6641-4a6f-bd30-748c69e115f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.446851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed56dec-6641-4a6f-bd30-748c69e115f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:11.447157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed56dec-6641-4a6f-bd30-748c69e115f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.448550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed56dec-6641-4a6f-bd30-748c69e115f2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:11.456761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 257/512] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:11.461785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed56dec-6641-4a6f-bd30-748c69e115f2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.464612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44677ac2-74a0-417a-a494-4d66ba33511c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.498511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf49bd-3ec8-42af-adf2-c1d71563101d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.499142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf49bd-3ec8-42af-adf2-c1d71563101d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.499445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf49bd-3ec8-42af-adf2-c1d71563101d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.499705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf49bd-3ec8-42af-adf2-c1d71563101d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.518654 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:11.562354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed56dec-6641-4a6f-bd30-748c69e115f2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.567218 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf49bd-3ec8-42af-adf2-c1d71563101d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.568676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf49bd-3ec8-42af-adf2-c1d71563101d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:11.596273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ede07c-313e-4721-9173-5a3e19b82133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.596883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede07c-313e-4721-9173-5a3e19b82133 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.597196 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede07c-313e-4721-9173-5a3e19b82133 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.597472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede07c-313e-4721-9173-5a3e19b82133 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.598160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ede07c-313e-4721-9173-5a3e19b82133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:11.598466 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ede07c-313e-4721-9173-5a3e19b82133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.599828 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ede07c-313e-4721-9173-5a3e19b82133 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:11.611445 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 258/513] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:11.612044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ede07c-313e-4721-9173-5a3e19b82133 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.616926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:11.670716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 256/514] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:11.671602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf49bd-3ec8-42af-adf2-c1d71563101d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.716687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ede07c-313e-4721-9173-5a3e19b82133 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.720416 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:11.751958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92508e24-30c9-4301-a65e-040a3d1d819c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.752578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92508e24-30c9-4301-a65e-040a3d1d819c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.752863 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92508e24-30c9-4301-a65e-040a3d1d819c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.753174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92508e24-30c9-4301-a65e-040a3d1d819c None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.753935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92508e24-30c9-4301-a65e-040a3d1d819c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:11.754231 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92508e24-30c9-4301-a65e-040a3d1d819c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.755661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92508e24-30c9-4301-a65e-040a3d1d819c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:11.769451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf49bd-3ec8-42af-adf2-c1d71563101d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.773551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 259/515] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:11.774381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92508e24-30c9-4301-a65e-040a3d1d819c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.804605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb12a1-30f9-4412-92f3-5b7e29bffd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.805259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb12a1-30f9-4412-92f3-5b7e29bffd2d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.805540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb12a1-30f9-4412-92f3-5b7e29bffd2d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.805818 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb12a1-30f9-4412-92f3-5b7e29bffd2d None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.806512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb12a1-30f9-4412-92f3-5b7e29bffd2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:11.806818 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb12a1-30f9-4412-92f3-5b7e29bffd2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.808181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb12a1-30f9-4412-92f3-5b7e29bffd2d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:11.819642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 257/516] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:11.820350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb12a1-30f9-4412-92f3-5b7e29bffd2d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.834857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:11.871673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92508e24-30c9-4301-a65e-040a3d1d819c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.905513 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9140cc-00fe-47ab-93f5-1d6c15134952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.906150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9140cc-00fe-47ab-93f5-1d6c15134952 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.906428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9140cc-00fe-47ab-93f5-1d6c15134952 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.906698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9140cc-00fe-47ab-93f5-1d6c15134952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.916418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9140cc-00fe-47ab-93f5-1d6c15134952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.917892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9140cc-00fe-47ab-93f5-1d6c15134952 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:11.918544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb12a1-30f9-4412-92f3-5b7e29bffd2d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:11.928558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ae9140cc-00fe-47ab-93f5-1d6c15134952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:11.932411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:11.933473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:11.945593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:11.945593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:11.945593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:11.945593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:11.945593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 260/517] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:11.945593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9140cc-00fe-47ab-93f5-1d6c15134952 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:11.951956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a812c-3758-4609-b8e4-a0ee0dc9dad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:11.953131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a812c-3758-4609-b8e4-a0ee0dc9dad0 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:11.953407 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a812c-3758-4609-b8e4-a0ee0dc9dad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:11.953676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a812c-3758-4609-b8e4-a0ee0dc9dad0 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:11.954350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a812c-3758-4609-b8e4-a0ee0dc9dad0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:11.955779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a812c-3758-4609-b8e4-a0ee0dc9dad0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.033708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9140cc-00fe-47ab-93f5-1d6c15134952 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.035976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 258/518] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:53:11 2022] POST /identity/v3/users => generated 247 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:12.036690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a812c-3758-4609-b8e4-a0ee0dc9dad0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:12.051320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:12.074510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a657bf-a5e5-4d4d-a921-4f16d40e8853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.075162 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a657bf-a5e5-4d4d-a921-4f16d40e8853 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.075616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a657bf-a5e5-4d4d-a921-4f16d40e8853 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.075933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a657bf-a5e5-4d4d-a921-4f16d40e8853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.090944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a657bf-a5e5-4d4d-a921-4f16d40e8853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.093667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a657bf-a5e5-4d4d-a921-4f16d40e8853 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-44a657bf-a5e5-4d4d-a921-4f16d40e8853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:12.115889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:12.121383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:12.122328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:12.122328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:12.122328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:12.122328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:12.122328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 261/519] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:12.122328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a657bf-a5e5-4d4d-a921-4f16d40e8853 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:12.164130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a812c-3758-4609-b8e4-a0ee0dc9dad0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.205626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42404a80-b31b-4ea7-b40d-d18f77e2cbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.206252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42404a80-b31b-4ea7-b40d-d18f77e2cbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.206528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42404a80-b31b-4ea7-b40d-d18f77e2cbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.207529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42404a80-b31b-4ea7-b40d-d18f77e2cbf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.207634 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42404a80-b31b-4ea7-b40d-d18f77e2cbf5 None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=7d7bcb4e7c434622b650cdfc500a9680) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:12.207880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42404a80-b31b-4ea7-b40d-d18f77e2cbf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.209341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42404a80-b31b-4ea7-b40d-d18f77e2cbf5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.215806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 259/520] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:12 2022] GET /identity/v3/role_assignments?user.id=7d7bcb4e7c434622b650cdfc500a9680&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:12.216529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42404a80-b31b-4ea7-b40d-d18f77e2cbf5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:12.256257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a657bf-a5e5-4d4d-a921-4f16d40e8853 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.258111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:12.300331 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f4a27-8d7d-4259-bfa2-0d47a6c8a7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.301406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f4a27-8d7d-4259-bfa2-0d47a6c8a7bb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.301825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f4a27-8d7d-4259-bfa2-0d47a6c8a7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.302285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f4a27-8d7d-4259-bfa2-0d47a6c8a7bb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.314849 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f4a27-8d7d-4259-bfa2-0d47a6c8a7bb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.316938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f4a27-8d7d-4259-bfa2-0d47a6c8a7bb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-6b0f4a27-8d7d-4259-bfa2-0d47a6c8a7bb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:12.335297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:12.336223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:12.337150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:12.337150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:12.337150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:12.337150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:12.337150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:12.337150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:12.337150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:12.337150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:12.337150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:53:12.337150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:12.337673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 262/521] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:12.338349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f4a27-8d7d-4259-bfa2-0d47a6c8a7bb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:12.354421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42404a80-b31b-4ea7-b40d-d18f77e2cbf5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.377969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:12.406266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87c58a-a2fd-438a-915b-8f26867bd7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.406953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87c58a-a2fd-438a-915b-8f26867bd7df None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.407176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87c58a-a2fd-438a-915b-8f26867bd7df None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.407452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87c58a-a2fd-438a-915b-8f26867bd7df None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.408157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87c58a-a2fd-438a-915b-8f26867bd7df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:12.408464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87c58a-a2fd-438a-915b-8f26867bd7df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.410584 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87c58a-a2fd-438a-915b-8f26867bd7df None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.431578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 260/522] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:12.432729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87c58a-a2fd-438a-915b-8f26867bd7df None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:12.473615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f4a27-8d7d-4259-bfa2-0d47a6c8a7bb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.510920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe1406-61d5-4a3f-8912-4c68ce72ad20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.511914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe1406-61d5-4a3f-8912-4c68ce72ad20 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.512454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe1406-61d5-4a3f-8912-4c68ce72ad20 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.512931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe1406-61d5-4a3f-8912-4c68ce72ad20 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.513922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe1406-61d5-4a3f-8912-4c68ce72ad20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:12.514427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe1406-61d5-4a3f-8912-4c68ce72ad20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.516121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe1406-61d5-4a3f-8912-4c68ce72ad20 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.528723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 263/523] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:12.529912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe1406-61d5-4a3f-8912-4c68ce72ad20 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:12.541969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:12.553238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87c58a-a2fd-438a-915b-8f26867bd7df None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.595353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812271-0807-4e1e-9196-e7143fa76dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.596126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812271-0807-4e1e-9196-e7143fa76dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.596413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812271-0807-4e1e-9196-e7143fa76dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.596692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812271-0807-4e1e-9196-e7143fa76dc7 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.597538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6812271-0807-4e1e-9196-e7143fa76dc7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:12.597958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6812271-0807-4e1e-9196-e7143fa76dc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.599497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6812271-0807-4e1e-9196-e7143fa76dc7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.620014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 261/524] 10.222.0.86 () {56 vars in 1137 bytes} [Thu Jul 28 17:53:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:12.621288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812271-0807-4e1e-9196-e7143fa76dc7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:12.631572 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:12.651653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe1406-61d5-4a3f-8912-4c68ce72ad20 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.701680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf90b8b-8280-4a84-9179-c88a9ecd5d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.701680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf90b8b-8280-4a84-9179-c88a9ecd5d53 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.701680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf90b8b-8280-4a84-9179-c88a9ecd5d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.701680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf90b8b-8280-4a84-9179-c88a9ecd5d53 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.712824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf90b8b-8280-4a84-9179-c88a9ecd5d53 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.714402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf90b8b-8280-4a84-9179-c88a9ecd5d53 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-bbf90b8b-8280-4a84-9179-c88a9ecd5d53 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:12.728516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:12.729690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:12.730752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:12.730752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:12.730752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:12.730752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:12.730752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:12.730752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:12.730752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:12.730752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:12.730752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:53:12.730752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:12.730752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 264/525] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:12.731341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf90b8b-8280-4a84-9179-c88a9ecd5d53 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:12.746028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812271-0807-4e1e-9196-e7143fa76dc7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.784534 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df00f6-f7a7-46f8-88b8-d688f2caa1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.785615 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df00f6-f7a7-46f8-88b8-d688f2caa1ae None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.786132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df00f6-f7a7-46f8-88b8-d688f2caa1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.786553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df00f6-f7a7-46f8-88b8-d688f2caa1ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.794163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df00f6-f7a7-46f8-88b8-d688f2caa1ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.796493 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df00f6-f7a7-46f8-88b8-d688f2caa1ae None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-14df00f6-f7a7-46f8-88b8-d688f2caa1ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:12.809713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:12.810881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:12.812453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:12.812453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:12.812453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:12.812453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:12.812453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:12.812453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:12.812453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:12.812453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:12.814591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 262/526] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:12.815605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df00f6-f7a7-46f8-88b8-d688f2caa1ae None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:12.827500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:12.841866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf90b8b-8280-4a84-9179-c88a9ecd5d53 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.878194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5693e69-56d8-431f-b644-38e11ffd0453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.879074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5693e69-56d8-431f-b644-38e11ffd0453 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.880142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5693e69-56d8-431f-b644-38e11ffd0453 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.880953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5693e69-56d8-431f-b644-38e11ffd0453 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.897906 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:12.923883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df00f6-f7a7-46f8-88b8-d688f2caa1ae None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.954691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5693e69-56d8-431f-b644-38e11ffd0453 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.956360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5693e69-56d8-431f-b644-38e11ffd0453 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.961725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fcfb8-316c-458f-878c-370c3abc24d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.962552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fcfb8-316c-458f-878c-370c3abc24d0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.962989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fcfb8-316c-458f-878c-370c3abc24d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.963412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fcfb8-316c-458f-878c-370c3abc24d0 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.964466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fcfb8-316c-458f-878c-370c3abc24d0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:12.964936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fcfb8-316c-458f-878c-370c3abc24d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.966663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fcfb8-316c-458f-878c-370c3abc24d0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.999707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 263/527] 10.222.0.86 () {56 vars in 1139 bytes} [Thu Jul 28 17:53:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:13.000866 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fcfb8-316c-458f-878c-370c3abc24d0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.070349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 265/528] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:13.071243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5693e69-56d8-431f-b644-38e11ffd0453 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.073318 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:13.110651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fcfb8-316c-458f-878c-370c3abc24d0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.121059 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:13.149181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76798a-b21f-443b-9116-0a1cb31dfa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.150135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76798a-b21f-443b-9116-0a1cb31dfa47 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.150628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76798a-b21f-443b-9116-0a1cb31dfa47 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.151089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76798a-b21f-443b-9116-0a1cb31dfa47 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.152011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76798a-b21f-443b-9116-0a1cb31dfa47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:13.152485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76798a-b21f-443b-9116-0a1cb31dfa47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.154117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76798a-b21f-443b-9116-0a1cb31dfa47 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.173560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 264/529] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:13.174782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76798a-b21f-443b-9116-0a1cb31dfa47 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.184039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5693e69-56d8-431f-b644-38e11ffd0453 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.222833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb06e7c-edae-49f9-9716-77092f5c686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.224479 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb06e7c-edae-49f9-9716-77092f5c686d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.224860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb06e7c-edae-49f9-9716-77092f5c686d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.225260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb06e7c-edae-49f9-9716-77092f5c686d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.309399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76798a-b21f-443b-9116-0a1cb31dfa47 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.313090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb06e7c-edae-49f9-9716-77092f5c686d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.314578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb06e7c-edae-49f9-9716-77092f5c686d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.338087 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:13.348246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539af4f-f1aa-4243-b7f6-84727d8ab042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.349362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539af4f-f1aa-4243-b7f6-84727d8ab042 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.349808 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539af4f-f1aa-4243-b7f6-84727d8ab042 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.350240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539af4f-f1aa-4243-b7f6-84727d8ab042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.362264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:13.363135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539af4f-f1aa-4243-b7f6-84727d8ab042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.364920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539af4f-f1aa-4243-b7f6-84727d8ab042 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-1539af4f-f1aa-4243-b7f6-84727d8ab042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:13.385169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:13.386095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:13.386930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:13.386930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:13.386930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:13.386930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:13.386930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 265/530] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:13.388164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539af4f-f1aa-4243-b7f6-84727d8ab042 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.432134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 266/531] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:13.433611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb06e7c-edae-49f9-9716-77092f5c686d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.514952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539af4f-f1aa-4243-b7f6-84727d8ab042 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.541446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb06e7c-edae-49f9-9716-77092f5c686d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.558017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2635f-22a8-49d8-966c-e84192866211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.558660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2635f-22a8-49d8-966c-e84192866211 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.558950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2635f-22a8-49d8-966c-e84192866211 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.559229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2635f-22a8-49d8-966c-e84192866211 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.560318 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2635f-22a8-49d8-966c-e84192866211 None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=c47f1552e73648ceb8a421c0543c2974) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:13.560635 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2635f-22a8-49d8-966c-e84192866211 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.562800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2635f-22a8-49d8-966c-e84192866211 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.570042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 266/532] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/role_assignments?user.id=c47f1552e73648ceb8a421c0543c2974&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:13.571181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2635f-22a8-49d8-966c-e84192866211 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.583190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006792f8-61a6-446e-a941-696aacb7a0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.584249 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006792f8-61a6-446e-a941-696aacb7a0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.584787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006792f8-61a6-446e-a941-696aacb7a0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.585385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006792f8-61a6-446e-a941-696aacb7a0b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.586327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006792f8-61a6-446e-a941-696aacb7a0b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:13.586935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006792f8-61a6-446e-a941-696aacb7a0b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.588703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006792f8-61a6-446e-a941-696aacb7a0b7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.603964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 267/533] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:13.610314 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006792f8-61a6-446e-a941-696aacb7a0b7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.621266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:13.626294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:13.697269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2635f-22a8-49d8-966c-e84192866211 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.740174 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87390c69-a7b7-414e-a676-4440b7d151ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.741285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87390c69-a7b7-414e-a676-4440b7d151ce None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.741720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87390c69-a7b7-414e-a676-4440b7d151ce None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.742436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87390c69-a7b7-414e-a676-4440b7d151ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.756250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87390c69-a7b7-414e-a676-4440b7d151ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.758398 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87390c69-a7b7-414e-a676-4440b7d151ce None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-87390c69-a7b7-414e-a676-4440b7d151ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:13.776113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:13.777339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:13.778214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:13.778214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:13.778214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:13.778214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:13.781424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 267/534] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:13.786703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87390c69-a7b7-414e-a676-4440b7d151ce None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.846608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006792f8-61a6-446e-a941-696aacb7a0b7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.891412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e26ce-8c73-42aa-a062-05433b00c4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.892305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e26ce-8c73-42aa-a062-05433b00c4fc None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.892739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e26ce-8c73-42aa-a062-05433b00c4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.901003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e26ce-8c73-42aa-a062-05433b00c4fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.904892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:13.913305 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87390c69-a7b7-414e-a676-4440b7d151ce None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.914250 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e26ce-8c73-42aa-a062-05433b00c4fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.915873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e26ce-8c73-42aa-a062-05433b00c4fc None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-704e26ce-8c73-42aa-a062-05433b00c4fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:13.934620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:13.935725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:13.936719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:13.936719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:13.936719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:13.936719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:13.936719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:13.936719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:13.936719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:13.936719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:13.945275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 268/535] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:13.949581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aabc14-4f70-4c6c-ac0b-e17773c726d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.950278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aabc14-4f70-4c6c-ac0b-e17773c726d4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.950563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aabc14-4f70-4c6c-ac0b-e17773c726d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.950836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aabc14-4f70-4c6c-ac0b-e17773c726d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.951539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aabc14-4f70-4c6c-ac0b-e17773c726d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:13.951844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aabc14-4f70-4c6c-ac0b-e17773c726d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.953292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aabc14-4f70-4c6c-ac0b-e17773c726d4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.953964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e26ce-8c73-42aa-a062-05433b00c4fc None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.971964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 268/536] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:13.975198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aabc14-4f70-4c6c-ac0b-e17773c726d4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:14.052424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:14.105011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e26ce-8c73-42aa-a062-05433b00c4fc None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:14.147020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aabc14-4f70-4c6c-ac0b-e17773c726d4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:14.170056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2483b-cb88-49e9-95fb-d60ee1ea87e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:14.171612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2483b-cb88-49e9-95fb-d60ee1ea87e4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.171893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2483b-cb88-49e9-95fb-d60ee1ea87e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.172163 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2483b-cb88-49e9-95fb-d60ee1ea87e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.172926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2483b-cb88-49e9-95fb-d60ee1ea87e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:14.173271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2483b-cb88-49e9-95fb-d60ee1ea87e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:14.175534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2483b-cb88-49e9-95fb-d60ee1ea87e4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:14.190023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 269/537] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:14.191241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188cdb5-284b-4f88-97db-1bc76633fa35 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.191529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188cdb5-284b-4f88-97db-1bc76633fa35 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.191806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188cdb5-284b-4f88-97db-1bc76633fa35 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.192631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcb4cf-8af3-4f03-bf4a-14aa79a1da0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:14.193437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcb4cf-8af3-4f03-bf4a-14aa79a1da0f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.197197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcb4cf-8af3-4f03-bf4a-14aa79a1da0f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.197500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcb4cf-8af3-4f03-bf4a-14aa79a1da0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.197889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 270/538] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:14.198933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188cdb5-284b-4f88-97db-1bc76633fa35 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:14.205681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcb4cf-8af3-4f03-bf4a-14aa79a1da0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:14.207066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcb4cf-8af3-4f03-bf4a-14aa79a1da0f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d9bcb4cf-8af3-4f03-bf4a-14aa79a1da0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:14.220187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:14.221389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:14.222238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:14.222238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:14.222238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:14.222238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:14.222238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:14.222238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:14.222238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:14.222238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:14.222238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 269/539] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:14.226260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed20f7-68ef-47be-be34-784a77d9d626 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.226260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed20f7-68ef-47be-be34-784a77d9d626 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.226260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed20f7-68ef-47be-be34-784a77d9d626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.244207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a3ed20f7-68ef-47be-be34-784a77d9d626 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:14.245409 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a3ed20f7-68ef-47be-be34-784a77d9d626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:14.257436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:14.337666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188cdb5-284b-4f88-97db-1bc76633fa35 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:14.353479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 270/540] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:14.353479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed20f7-68ef-47be-be34-784a77d9d626 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:14.391303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:14.404635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa482233-8e4c-403a-bfb0-67659d589468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:14.406183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa482233-8e4c-403a-bfb0-67659d589468 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.406665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa482233-8e4c-403a-bfb0-67659d589468 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.407146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa482233-8e4c-403a-bfb0-67659d589468 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.408133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa482233-8e4c-403a-bfb0-67659d589468 None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=9dbaf1e343474322b2567e2dfda99d4e) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:14.408699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa482233-8e4c-403a-bfb0-67659d589468 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:14.410509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa482233-8e4c-403a-bfb0-67659d589468 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:14.419373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 271/541] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:14 2022] GET /identity/v3/role_assignments?user.id=9dbaf1e343474322b2567e2dfda99d4e&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:14.420186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa482233-8e4c-403a-bfb0-67659d589468 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:14.490142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed20f7-68ef-47be-be34-784a77d9d626 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:14.530000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78170e5-c8fe-4bf9-95c8-f6e718d2db93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:14.530688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78170e5-c8fe-4bf9-95c8-f6e718d2db93 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.530969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78170e5-c8fe-4bf9-95c8-f6e718d2db93 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.531245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78170e5-c8fe-4bf9-95c8-f6e718d2db93 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.532021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78170e5-c8fe-4bf9-95c8-f6e718d2db93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:14.532329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78170e5-c8fe-4bf9-95c8-f6e718d2db93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:14.533918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78170e5-c8fe-4bf9-95c8-f6e718d2db93 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:14.547577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 271/542] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:14.553691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcbc99-9ab0-4146-88ef-74d246a58533 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.553979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcbc99-9ab0-4146-88ef-74d246a58533 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.554262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcbc99-9ab0-4146-88ef-74d246a58533 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.559959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 272/543] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:14.560898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcbc99-9ab0-4146-88ef-74d246a58533 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:14.566467 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:14.619952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa482233-8e4c-403a-bfb0-67659d589468 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:14.682205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3581a82-7568-4aed-a0c0-c8ea1e613892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:14.682861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3581a82-7568-4aed-a0c0-c8ea1e613892 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.683141 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3581a82-7568-4aed-a0c0-c8ea1e613892 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.683413 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3581a82-7568-4aed-a0c0-c8ea1e613892 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.693231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcbc99-9ab0-4146-88ef-74d246a58533 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:14.695740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3581a82-7568-4aed-a0c0-c8ea1e613892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c3581a82-7568-4aed-a0c0-c8ea1e613892 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:14.700633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:14.701618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:14.702475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 272/544] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:14.703670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55545e32-de50-47fc-8709-d2fb9d584de2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.703767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55545e32-de50-47fc-8709-d2fb9d584de2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.704043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55545e32-de50-47fc-8709-d2fb9d584de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.723143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-55545e32-de50-47fc-8709-d2fb9d584de2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:14.724334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-55545e32-de50-47fc-8709-d2fb9d584de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:14.740504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a5db8-2032-4d67-abec-0e27040b6274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:14.741483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a5db8-2032-4d67-abec-0e27040b6274 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.742202 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a5db8-2032-4d67-abec-0e27040b6274 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.743118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a5db8-2032-4d67-abec-0e27040b6274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.744412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a5db8-2032-4d67-abec-0e27040b6274 None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=b038336582a0435dbd47ad2c09cd23e0) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:14.744928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a5db8-2032-4d67-abec-0e27040b6274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:14.747314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a5db8-2032-4d67-abec-0e27040b6274 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:14.755280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 273/545] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:14 2022] GET /identity/v3/role_assignments?user.id=b038336582a0435dbd47ad2c09cd23e0&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:14.756290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a5db8-2032-4d67-abec-0e27040b6274 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:14.801397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:14.854840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 273/546] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:14.857627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:14.886504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55545e32-de50-47fc-8709-d2fb9d584de2 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:14.901595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a5db8-2032-4d67-abec-0e27040b6274 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:14.946633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00249-484f-4855-a5d5-276b3c0b1aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:14.947719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00249-484f-4855-a5d5-276b3c0b1aec None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:14.948220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00249-484f-4855-a5d5-276b3c0b1aec None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:14.948680 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00249-484f-4855-a5d5-276b3c0b1aec None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:14.949670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b00249-484f-4855-a5d5-276b3c0b1aec None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:14.950495 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b00249-484f-4855-a5d5-276b3c0b1aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:14.952877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b00249-484f-4855-a5d5-276b3c0b1aec None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:14.963829 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 274/547] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:14.969663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00249-484f-4855-a5d5-276b3c0b1aec None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:15.045639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55545e32-de50-47fc-8709-d2fb9d584de2 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:15.100397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a571b20-df6c-44f2-ac1b-ce8fad65b96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:15.101377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a571b20-df6c-44f2-ac1b-ce8fad65b96e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:15.101865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a571b20-df6c-44f2-ac1b-ce8fad65b96e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:15.102316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a571b20-df6c-44f2-ac1b-ce8fad65b96e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:15.114552 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a571b20-df6c-44f2-ac1b-ce8fad65b96e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-7a571b20-df6c-44f2-ac1b-ce8fad65b96e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:15.120785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:15.121798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:15.122594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:15.126265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 274/548] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:15.132931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a571b20-df6c-44f2-ac1b-ce8fad65b96e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:15.162185 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00249-484f-4855-a5d5-276b3c0b1aec None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:15.208778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:15.217320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:15.224696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbded0d-9cb2-4d5a-800d-6de2c09d8feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:15.225494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbded0d-9cb2-4d5a-800d-6de2c09d8feb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:15.225804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbded0d-9cb2-4d5a-800d-6de2c09d8feb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:15.226078 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbded0d-9cb2-4d5a-800d-6de2c09d8feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:15.240891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbded0d-9cb2-4d5a-800d-6de2c09d8feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:15.242378 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbded0d-9cb2-4d5a-800d-6de2c09d8feb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-9dbded0d-9cb2-4d5a-800d-6de2c09d8feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:15.266951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:15.268036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:15.268950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:15.268950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:15.268950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:15.268950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:15.268950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 275/549] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:15.281742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbded0d-9cb2-4d5a-800d-6de2c09d8feb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:15.314975 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a571b20-df6c-44f2-ac1b-ce8fad65b96e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:15.381177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dbb90-1d97-4267-ada0-3bdc03cdf538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:15.382116 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dbb90-1d97-4267-ada0-3bdc03cdf538 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:15.382608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dbb90-1d97-4267-ada0-3bdc03cdf538 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:15.383246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dbb90-1d97-4267-ada0-3bdc03cdf538 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:15.384168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dbb90-1d97-4267-ada0-3bdc03cdf538 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:15.386241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dbb90-1d97-4267-ada0-3bdc03cdf538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:15.387810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dbb90-1d97-4267-ada0-3bdc03cdf538 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:15.418807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 275/550] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:15.428576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dbb90-1d97-4267-ada0-3bdc03cdf538 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:15.465777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbded0d-9cb2-4d5a-800d-6de2c09d8feb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:15.534139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bfbe6-b41d-463e-8a18-f80b5f7f0fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:15.536478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bfbe6-b41d-463e-8a18-f80b5f7f0fab None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:15.537201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bfbe6-b41d-463e-8a18-f80b5f7f0fab None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:15.538782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bfbe6-b41d-463e-8a18-f80b5f7f0fab None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:15.539673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bfbe6-b41d-463e-8a18-f80b5f7f0fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:15.541126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bfbe6-b41d-463e-8a18-f80b5f7f0fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:15.542817 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bfbe6-b41d-463e-8a18-f80b5f7f0fab None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:15.574081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 276/551] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:15.575756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bfbe6-b41d-463e-8a18-f80b5f7f0fab None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:15.629474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:15.653684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dbb90-1d97-4267-ada0-3bdc03cdf538 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:15.665431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:15.742931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8895021-868f-4755-8fcd-e15e5c4459a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:15.743602 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8895021-868f-4755-8fcd-e15e5c4459a5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:15.743887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8895021-868f-4755-8fcd-e15e5c4459a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:15.744164 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8895021-868f-4755-8fcd-e15e5c4459a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:15.761680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8895021-868f-4755-8fcd-e15e5c4459a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:15.763796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8895021-868f-4755-8fcd-e15e5c4459a5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:15.772989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bfbe6-b41d-463e-8a18-f80b5f7f0fab None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a8895021-868f-4755-8fcd-e15e5c4459a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:15.788934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:15.789877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:15.790788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:15.790788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:15.790788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:15.790788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:15.793154 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 276/552] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:15.802055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8895021-868f-4755-8fcd-e15e5c4459a5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:15.861112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cb426-ccff-4d44-a936-bcd387fc6339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:15.861765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cb426-ccff-4d44-a936-bcd387fc6339 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:15.862047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cb426-ccff-4d44-a936-bcd387fc6339 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:15.862322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cb426-ccff-4d44-a936-bcd387fc6339 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:15.886228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cb426-ccff-4d44-a936-bcd387fc6339 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:15.887575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cb426-ccff-4d44-a936-bcd387fc6339 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-1e5cb426-ccff-4d44-a936-bcd387fc6339 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:15.911118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:15.912183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:15.913148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:15.913148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:15.913148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:15.913148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:15.913148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:15.913148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:15.913148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:15.913148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:15.913148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:15.913148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:15.914820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8895021-868f-4755-8fcd-e15e5c4459a5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:15.921316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 277/553] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:15.922190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cb426-ccff-4d44-a936-bcd387fc6339 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:15.957377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03910b-c653-494d-97f5-1310cbd27185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:15.960808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03910b-c653-494d-97f5-1310cbd27185 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:15.963566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03910b-c653-494d-97f5-1310cbd27185 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:15.964323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03910b-c653-494d-97f5-1310cbd27185 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:15.965306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03910b-c653-494d-97f5-1310cbd27185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:15.966016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03910b-c653-494d-97f5-1310cbd27185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:15.969076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03910b-c653-494d-97f5-1310cbd27185 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:15.990045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 277/554] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:15.991340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765547-af7e-451e-9cad-db080887af7d None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:15.991621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765547-af7e-451e-9cad-db080887af7d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:15.991900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765547-af7e-451e-9cad-db080887af7d None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:15.997101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 278/555] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:15.999763 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3765547-af7e-451e-9cad-db080887af7d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:16.018595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:16.072667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cb426-ccff-4d44-a936-bcd387fc6339 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:16.075486 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:16.110877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3765547-af7e-451e-9cad-db080887af7d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:16.114122 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35342404-5f34-4367-b292-1646097b1c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:16.115549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35342404-5f34-4367-b292-1646097b1c17 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:16.121633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35342404-5f34-4367-b292-1646097b1c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:16.122095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35342404-5f34-4367-b292-1646097b1c17 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:16.123054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35342404-5f34-4367-b292-1646097b1c17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:16.123441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35342404-5f34-4367-b292-1646097b1c17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:16.126269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35342404-5f34-4367-b292-1646097b1c17 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:16.156345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 278/556] 10.222.0.86 () {56 vars in 1135 bytes} [Thu Jul 28 17:53:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:16.156345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b05160-13ec-4477-9670-4a8c3242cbd8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:16.156345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b05160-13ec-4477-9670-4a8c3242cbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:16.156345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b05160-13ec-4477-9670-4a8c3242cbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:16.159544 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce22de-73ae-4b7c-b31c-65214e7a7fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:16.160369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce22de-73ae-4b7c-b31c-65214e7a7fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:16.160896 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce22de-73ae-4b7c-b31c-65214e7a7fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:16.161331 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce22de-73ae-4b7c-b31c-65214e7a7fb3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:16.170198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce22de-73ae-4b7c-b31c-65214e7a7fb3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:16.172459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce22de-73ae-4b7c-b31c-65214e7a7fb3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:16.175060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b7b05160-13ec-4477-9670-4a8c3242cbd8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:16.176426 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b7b05160-13ec-4477-9670-4a8c3242cbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-23ce22de-73ae-4b7c-b31c-65214e7a7fb3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:16.185178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:16.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:16.187709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:16.187709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:16.187709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:16.187709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:16.187709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:16.187709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:16.187709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:16.187709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:16.187709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:16.187709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:16.187709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 279/557] 10.222.0.86 () {56 vars in 1093 bytes} [Thu Jul 28 17:53:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:16.188298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce22de-73ae-4b7c-b31c-65214e7a7fb3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:16.268194 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 279/558] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:16.269408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b05160-13ec-4477-9670-4a8c3242cbd8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:16.280930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:16.309901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce22de-73ae-4b7c-b31c-65214e7a7fb3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:16.349905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35694d-2eab-46e9-97fc-609740ff7112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:16.350672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35694d-2eab-46e9-97fc-609740ff7112 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:16.351003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35694d-2eab-46e9-97fc-609740ff7112 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:16.351358 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb35694d-2eab-46e9-97fc-609740ff7112 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:16.396846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:16.407592 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b05160-13ec-4477-9670-4a8c3242cbd8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:16.447659 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35694d-2eab-46e9-97fc-609740ff7112 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:16.449208 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb35694d-2eab-46e9-97fc-609740ff7112 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:16.459821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3945c-0011-4efb-bc21-ae18b7e40f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:16.461107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3945c-0011-4efb-bc21-ae18b7e40f5c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:16.461367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3945c-0011-4efb-bc21-ae18b7e40f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:16.461643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3945c-0011-4efb-bc21-ae18b7e40f5c None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:16.462457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3945c-0011-4efb-bc21-ae18b7e40f5c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:16.462763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3945c-0011-4efb-bc21-ae18b7e40f5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:16.464309 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3945c-0011-4efb-bc21-ae18b7e40f5c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:16.488188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 280/559] 10.222.0.86 () {56 vars in 1133 bytes} [Thu Jul 28 17:53:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:16.489022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3945c-0011-4efb-bc21-ae18b7e40f5c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:16.543434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:16.575536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 280/560] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:16.575536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35694d-2eab-46e9-97fc-609740ff7112 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:16.626085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3945c-0011-4efb-bc21-ae18b7e40f5c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:16.672574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290df72a-d403-4ff8-9348-fefe843a58e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:16.673783 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290df72a-d403-4ff8-9348-fefe843a58e9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:16.674217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290df72a-d403-4ff8-9348-fefe843a58e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:16.674794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290df72a-d403-4ff8-9348-fefe843a58e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:16.678356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb35694d-2eab-46e9-97fc-609740ff7112 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:16.691326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290df72a-d403-4ff8-9348-fefe843a58e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-290df72a-d403-4ff8-9348-fefe843a58e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:16.698014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:16.698957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:16.699814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 281/561] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:16.700507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290df72a-d403-4ff8-9348-fefe843a58e9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:16.715443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:16.721872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba30be1-47da-49d6-9661-a318e9c72905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:16.722524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba30be1-47da-49d6-9661-a318e9c72905 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:16.722811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba30be1-47da-49d6-9661-a318e9c72905 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:16.723090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba30be1-47da-49d6-9661-a318e9c72905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:16.733883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba30be1-47da-49d6-9661-a318e9c72905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:16.735395 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba30be1-47da-49d6-9661-a318e9c72905 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-aba30be1-47da-49d6-9661-a318e9c72905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:16.751952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:16.752880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:16.753813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:16.753813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:16.753813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:16.753813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:16.753813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 281/562] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:16.754088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba30be1-47da-49d6-9661-a318e9c72905 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:16.783594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:16.818093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290df72a-d403-4ff8-9348-fefe843a58e9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:16.855303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525382b-a8d5-4a3a-bba8-9758b9137986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:16.856515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525382b-a8d5-4a3a-bba8-9758b9137986 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:16.856808 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525382b-a8d5-4a3a-bba8-9758b9137986 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:16.857107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525382b-a8d5-4a3a-bba8-9758b9137986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:16.858310 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba30be1-47da-49d6-9661-a318e9c72905 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:16.868283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525382b-a8d5-4a3a-bba8-9758b9137986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:16.869878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525382b-a8d5-4a3a-bba8-9758b9137986 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d525382b-a8d5-4a3a-bba8-9758b9137986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:16.885411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:16.886330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:16.887222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:16.887222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:16.887222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:16.887222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:16.887222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 282/563] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:16.888204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e07e2-df54-41b4-bbb2-47b762b748a4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:16.888204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e07e2-df54-41b4-bbb2-47b762b748a4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:16.888377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e07e2-df54-41b4-bbb2-47b762b748a4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:16.890290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 283/564] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:16.890900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e07e2-df54-41b4-bbb2-47b762b748a4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:16.903383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e968045-3482-4826-8b14-2d581a2af645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:16.904014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e968045-3482-4826-8b14-2d581a2af645 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:16.904302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e968045-3482-4826-8b14-2d581a2af645 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:16.904679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e968045-3482-4826-8b14-2d581a2af645 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:16.905953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e968045-3482-4826-8b14-2d581a2af645 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:16.906263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e968045-3482-4826-8b14-2d581a2af645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:16.907662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e968045-3482-4826-8b14-2d581a2af645 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:16.920238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 282/565] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:16.920238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f0d4d-5f18-468c-87aa-afb9419f9adb None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:16.920238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f0d4d-5f18-468c-87aa-afb9419f9adb None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:16.920238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f0d4d-5f18-468c-87aa-afb9419f9adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:16.936841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-715f0d4d-5f18-468c-87aa-afb9419f9adb None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:16.938025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-715f0d4d-5f18-468c-87aa-afb9419f9adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:16.960479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:17.005377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e07e2-df54-41b4-bbb2-47b762b748a4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.008336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 283/566] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:17.009261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f0d4d-5f18-468c-87aa-afb9419f9adb None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.029983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:17.052781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06a009-5cdb-46ff-8442-8c92a569a4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.053436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06a009-5cdb-46ff-8442-8c92a569a4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.053719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06a009-5cdb-46ff-8442-8c92a569a4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.053995 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06a009-5cdb-46ff-8442-8c92a569a4b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.054781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06a009-5cdb-46ff-8442-8c92a569a4b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:17.055089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06a009-5cdb-46ff-8442-8c92a569a4b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:17.056463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06a009-5cdb-46ff-8442-8c92a569a4b4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:17.069135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 284/567] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:17.069950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06a009-5cdb-46ff-8442-8c92a569a4b4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.120009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f0d4d-5f18-468c-87aa-afb9419f9adb None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.155170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2047d-391b-476c-9908-54f71bd33aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.155798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2047d-391b-476c-9908-54f71bd33aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.156076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2047d-391b-476c-9908-54f71bd33aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.156372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2047d-391b-476c-9908-54f71bd33aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.157106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2047d-391b-476c-9908-54f71bd33aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:17.157414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2047d-391b-476c-9908-54f71bd33aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:17.158779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2047d-391b-476c-9908-54f71bd33aa4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:17.169933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 284/568] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:17.170731 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2047d-391b-476c-9908-54f71bd33aa4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.170837 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06a009-5cdb-46ff-8442-8c92a569a4b4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.191549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:17.205731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8c170-3cf0-436d-a7af-f6755be514a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.206366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8c170-3cf0-436d-a7af-f6755be514a5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.206646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8c170-3cf0-436d-a7af-f6755be514a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.206922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8c170-3cf0-436d-a7af-f6755be514a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.216689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8c170-3cf0-436d-a7af-f6755be514a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:17.218300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8c170-3cf0-436d-a7af-f6755be514a5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e2f8c170-3cf0-436d-a7af-f6755be514a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:17.233561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:17.234458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:17.235403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:17.235403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:17.235403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:17.235403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:17.235403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 285/569] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:17.235687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8c170-3cf0-436d-a7af-f6755be514a5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.251775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:17.273010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2047d-391b-476c-9908-54f71bd33aa4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.306644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb55777-9492-4095-9243-1b3087e5853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.307282 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb55777-9492-4095-9243-1b3087e5853d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.307563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb55777-9492-4095-9243-1b3087e5853d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.307837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb55777-9492-4095-9243-1b3087e5853d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.318987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb55777-9492-4095-9243-1b3087e5853d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-9cb55777-9492-4095-9243-1b3087e5853d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:17.324244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:17.325402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:17.326332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 285/570] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:17.327049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb4bfb-3ca8-492c-89a8-d93d9d579f3b None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.327418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb4bfb-3ca8-492c-89a8-d93d9d579f3b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.327698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb4bfb-3ca8-492c-89a8-d93d9d579f3b None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.332890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 286/571] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:17.332890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb4bfb-3ca8-492c-89a8-d93d9d579f3b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.343331 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8c170-3cf0-436d-a7af-f6755be514a5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.377919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce26b94-a679-477d-88be-4dabb91cfc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.378559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce26b94-a679-477d-88be-4dabb91cfc41 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.378839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce26b94-a679-477d-88be-4dabb91cfc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.379115 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce26b94-a679-477d-88be-4dabb91cfc41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.385944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce26b94-a679-477d-88be-4dabb91cfc41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:17.387365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce26b94-a679-477d-88be-4dabb91cfc41 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-5ce26b94-a679-477d-88be-4dabb91cfc41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:17.398608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:17.399742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:17.400858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:17.400858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:17.400858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:17.400858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:17.400858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:17.400858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:17.400858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:17.400858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:17.404354 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 286/572] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:17.405228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce26b94-a679-477d-88be-4dabb91cfc41 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.431938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb4bfb-3ca8-492c-89a8-d93d9d579f3b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.441415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:17.465805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f40af5-89dc-4bc6-83be-44fab820ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.466380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f40af5-89dc-4bc6-83be-44fab820ddfe None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.466661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f40af5-89dc-4bc6-83be-44fab820ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.466935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f40af5-89dc-4bc6-83be-44fab820ddfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.473612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f40af5-89dc-4bc6-83be-44fab820ddfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:17.475045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f40af5-89dc-4bc6-83be-44fab820ddfe None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:17.476731 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b1f40af5-89dc-4bc6-83be-44fab820ddfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:17.487108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:17.492217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:17.493156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:17.493156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:17.493156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:17.493156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:17.493156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:17.493156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:17.493156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:17.493156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:17.493156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 287/573] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:17.493156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f40af5-89dc-4bc6-83be-44fab820ddfe None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.514841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce26b94-a679-477d-88be-4dabb91cfc41 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.554785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f5738-59b3-4d1c-aa66-91f914116578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.555408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f5738-59b3-4d1c-aa66-91f914116578 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.555694 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f5738-59b3-4d1c-aa66-91f914116578 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.555974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f5738-59b3-4d1c-aa66-91f914116578 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.556668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f5738-59b3-4d1c-aa66-91f914116578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:17.557061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f5738-59b3-4d1c-aa66-91f914116578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:17.558432 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f5738-59b3-4d1c-aa66-91f914116578 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:17.580111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 287/574] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:17.580111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dde297-b6a0-42b3-87c8-295c3d92303c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.580111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dde297-b6a0-42b3-87c8-295c3d92303c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.580111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dde297-b6a0-42b3-87c8-295c3d92303c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.598521 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f40af5-89dc-4bc6-83be-44fab820ddfe None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.602532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-37dde297-b6a0-42b3-87c8-295c3d92303c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:17.603824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-37dde297-b6a0-42b3-87c8-295c3d92303c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:17.632373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367b617-69c1-463c-bb24-9e571ec688f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.633020 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367b617-69c1-463c-bb24-9e571ec688f2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.633304 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367b617-69c1-463c-bb24-9e571ec688f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.633580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367b617-69c1-463c-bb24-9e571ec688f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.634292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367b617-69c1-463c-bb24-9e571ec688f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:17.634597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367b617-69c1-463c-bb24-9e571ec688f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:17.635940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367b617-69c1-463c-bb24-9e571ec688f2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:17.643734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 288/575] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:17.644570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367b617-69c1-463c-bb24-9e571ec688f2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.670169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 288/576] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:17.671553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dde297-b6a0-42b3-87c8-295c3d92303c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.697005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:17.714035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:17.754526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367b617-69c1-463c-bb24-9e571ec688f2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.771241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dde297-b6a0-42b3-87c8-295c3d92303c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.787074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc7dd7-9c96-4e38-9d81-753a6383cdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.787697 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc7dd7-9c96-4e38-9d81-753a6383cdb2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.787981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc7dd7-9c96-4e38-9d81-753a6383cdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.788265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc7dd7-9c96-4e38-9d81-753a6383cdb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.789042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc7dd7-9c96-4e38-9d81-753a6383cdb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:17.789352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc7dd7-9c96-4e38-9d81-753a6383cdb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:17.790772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc7dd7-9c96-4e38-9d81-753a6383cdb2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:17.803539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 289/577] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:17.804158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc7dd7-9c96-4e38-9d81-753a6383cdb2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.805313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1a52a-b252-407a-95c7-20a0c64ee531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.805937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1a52a-b252-407a-95c7-20a0c64ee531 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.806219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1a52a-b252-407a-95c7-20a0c64ee531 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.806496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1a52a-b252-407a-95c7-20a0c64ee531 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.807245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1a52a-b252-407a-95c7-20a0c64ee531 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:17.807558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1a52a-b252-407a-95c7-20a0c64ee531 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:17.809104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1a52a-b252-407a-95c7-20a0c64ee531 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:17.820533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 289/578] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:17.821501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1a52a-b252-407a-95c7-20a0c64ee531 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.903488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:17.903694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc7dd7-9c96-4e38-9d81-753a6383cdb2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.921841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1a52a-b252-407a-95c7-20a0c64ee531 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:17.924951 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:17.938222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1531ba6-0c90-4136-8f7c-168184cc4ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.938858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1531ba6-0c90-4136-8f7c-168184cc4ade None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.939140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1531ba6-0c90-4136-8f7c-168184cc4ade None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.939418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1531ba6-0c90-4136-8f7c-168184cc4ade None admin] PATH_INFO: `/v3/users/glance` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.946512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1531ba6-0c90-4136-8f7c-168184cc4ade None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:17.947944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1531ba6-0c90-4136-8f7c-168184cc4ade None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:17.954522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd1dfa-4e7f-485d-83d8-0b1cf5d2a126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:17.955153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd1dfa-4e7f-485d-83d8-0b1cf5d2a126 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:17.955434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd1dfa-4e7f-485d-83d8-0b1cf5d2a126 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:17.955708 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd1dfa-4e7f-485d-83d8-0b1cf5d2a126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-e1531ba6-0c90-4136-8f7c-168184cc4ade None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:17.959459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:17.960589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 290/579] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:17.961622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1531ba6-0c90-4136-8f7c-168184cc4ade None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:17.967084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd1dfa-4e7f-485d-83d8-0b1cf5d2a126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:17.969175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd1dfa-4e7f-485d-83d8-0b1cf5d2a126 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-51fd1dfa-4e7f-485d-83d8-0b1cf5d2a126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:17.983862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:17.984861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:17.985775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:17.985775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:17.985775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:17.985775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:17.986344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 290/580] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:17.987241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd1dfa-4e7f-485d-83d8-0b1cf5d2a126 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:18.062780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1531ba6-0c90-4136-8f7c-168184cc4ade None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:18.083993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd1dfa-4e7f-485d-83d8-0b1cf5d2a126 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:18.096336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88002247-2177-49a9-b21b-be744c9719b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:18.097123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88002247-2177-49a9-b21b-be744c9719b1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.097341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88002247-2177-49a9-b21b-be744c9719b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.097613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88002247-2177-49a9-b21b-be744c9719b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.107421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88002247-2177-49a9-b21b-be744c9719b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-88002247-2177-49a9-b21b-be744c9719b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:18.112703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:18.113816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:18.114834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:18.118661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 291/581] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:18.119091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dd9fa-5286-4cf2-8831-859e3b45578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:18.119472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88002247-2177-49a9-b21b-be744c9719b1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:18.119865 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dd9fa-5286-4cf2-8831-859e3b45578d None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.120153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dd9fa-5286-4cf2-8831-859e3b45578d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.120430 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dd9fa-5286-4cf2-8831-859e3b45578d None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/7d7bcb4e7c434622b650cdfc500a9680/roles/ce68cefcc9b14645bbb5a2b411a23a44` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.122932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:18.135691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dd9fa-5286-4cf2-8831-859e3b45578d None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=7d7bcb4e7c434622b650cdfc500a9680, role_id=ce68cefcc9b14645bbb5a2b411a23a44)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:18.137174 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dd9fa-5286-4cf2-8831-859e3b45578d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:18.160788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:18.210904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 291/582] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:53:17 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/7d7bcb4e7c434622b650cdfc500a9680/roles/ce68cefcc9b14645bbb5a2b411a23a44 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:18.211712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dd9fa-5286-4cf2-8831-859e3b45578d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:18.226464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88002247-2177-49a9-b21b-be744c9719b1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:18.267111 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1885d-e7f5-4d1b-95e8-434268c9d4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:18.267794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1885d-e7f5-4d1b-95e8-434268c9d4cd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.268078 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1885d-e7f5-4d1b-95e8-434268c9d4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.268359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1885d-e7f5-4d1b-95e8-434268c9d4cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.269213 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1885d-e7f5-4d1b-95e8-434268c9d4cd None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=3077d844257c48b3b4ac2db9177fb6df) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:18.269531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1885d-e7f5-4d1b-95e8-434268c9d4cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:18.271020 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1885d-e7f5-4d1b-95e8-434268c9d4cd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:18.277547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 292/583] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:18 2022] GET /identity/v3/role_assignments?user.id=3077d844257c48b3b4ac2db9177fb6df&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:18.278216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1885d-e7f5-4d1b-95e8-434268c9d4cd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:18.353968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dd9fa-5286-4cf2-8831-859e3b45578d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:18.409055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104362a0-c0a3-473c-985e-193457dad980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:18.410294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104362a0-c0a3-473c-985e-193457dad980 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.410575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104362a0-c0a3-473c-985e-193457dad980 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.410852 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104362a0-c0a3-473c-985e-193457dad980 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.411683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104362a0-c0a3-473c-985e-193457dad980 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:18.411996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104362a0-c0a3-473c-985e-193457dad980 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:18.419120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104362a0-c0a3-473c-985e-193457dad980 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:18.430159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1885d-e7f5-4d1b-95e8-434268c9d4cd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:18.433162 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:18.440322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 292/584] 10.222.0.86 () {56 vars in 1137 bytes} [Thu Jul 28 17:53:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:18.441422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104362a0-c0a3-473c-985e-193457dad980 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:18.477307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920b208-557e-42f1-8bd7-f0637fced8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:18.478026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920b208-557e-42f1-8bd7-f0637fced8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.478316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920b208-557e-42f1-8bd7-f0637fced8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.478601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920b208-557e-42f1-8bd7-f0637fced8f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.479312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920b208-557e-42f1-8bd7-f0637fced8f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:18.479622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920b208-557e-42f1-8bd7-f0637fced8f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:18.481567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920b208-557e-42f1-8bd7-f0637fced8f8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:18.495729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:18.500191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 293/585] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:18.501596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920b208-557e-42f1-8bd7-f0637fced8f8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:18.552558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104362a0-c0a3-473c-985e-193457dad980 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:18.591050 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a465429-53c0-4e13-af5b-f885f1300537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:18.591785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a465429-53c0-4e13-af5b-f885f1300537 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.592171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a465429-53c0-4e13-af5b-f885f1300537 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.592527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a465429-53c0-4e13-af5b-f885f1300537 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.593733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a465429-53c0-4e13-af5b-f885f1300537 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:18.594154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a465429-53c0-4e13-af5b-f885f1300537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:18.595522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a465429-53c0-4e13-af5b-f885f1300537 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:18.608878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 293/586] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:18.608878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a465429-53c0-4e13-af5b-f885f1300537 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:18.636260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920b208-557e-42f1-8bd7-f0637fced8f8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:18.671610 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f1bdd-a347-41ac-8c31-61fc9011cfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:18.672365 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f1bdd-a347-41ac-8c31-61fc9011cfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.672602 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f1bdd-a347-41ac-8c31-61fc9011cfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.672826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f1bdd-a347-41ac-8c31-61fc9011cfa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.683327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f1bdd-a347-41ac-8c31-61fc9011cfa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:18.684548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f1bdd-a347-41ac-8c31-61fc9011cfa3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-915f1bdd-a347-41ac-8c31-61fc9011cfa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:18.700273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:18.701323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:18.702782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:18.702782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:18.702782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:18.702782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:18.702782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 294/587] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:18.703517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f1bdd-a347-41ac-8c31-61fc9011cfa3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:18.722213 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:18.731669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a465429-53c0-4e13-af5b-f885f1300537 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:18.734849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:18.774362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2ec24-9a73-4ce4-88a0-98e197cb9d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:18.775581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2ec24-9a73-4ce4-88a0-98e197cb9d73 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.775877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2ec24-9a73-4ce4-88a0-98e197cb9d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.776151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2ec24-9a73-4ce4-88a0-98e197cb9d73 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.786093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2ec24-9a73-4ce4-88a0-98e197cb9d73 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:18.787809 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2ec24-9a73-4ce4-88a0-98e197cb9d73 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-5aa2ec24-9a73-4ce4-88a0-98e197cb9d73 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:18.803424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:18.804422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:18.805436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:18.805436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:18.805436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:18.805436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:18.805436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:18.805436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:18.805436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:18.805436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:18.805436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:53:18.805436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:18.805436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 294/588] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:18.806033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2ec24-9a73-4ce4-88a0-98e197cb9d73 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:18.822300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f1bdd-a347-41ac-8c31-61fc9011cfa3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:18.862638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acce34-7202-4543-8081-f152f0720f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:18.863793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acce34-7202-4543-8081-f152f0720f42 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.864292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acce34-7202-4543-8081-f152f0720f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.864849 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acce34-7202-4543-8081-f152f0720f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.878681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acce34-7202-4543-8081-f152f0720f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:18.880508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acce34-7202-4543-8081-f152f0720f42 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-15acce34-7202-4543-8081-f152f0720f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:18.898199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:18.899142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:18.900188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:18.900188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:18.900188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:18.900188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:18.907221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 295/589] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:18.908727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acce34-7202-4543-8081-f152f0720f42 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:18.929262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2ec24-9a73-4ce4-88a0-98e197cb9d73 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:18.971327 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dd097-1f73-4872-86a7-c5a38849d4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:18.971950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dd097-1f73-4872-86a7-c5a38849d4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:18.972231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dd097-1f73-4872-86a7-c5a38849d4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:18.972502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dd097-1f73-4872-86a7-c5a38849d4f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:18.973230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dd097-1f73-4872-86a7-c5a38849d4f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:18.973535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dd097-1f73-4872-86a7-c5a38849d4f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:18.974894 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dd097-1f73-4872-86a7-c5a38849d4f0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:18.990839 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:18.991365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 295/590] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:18.992175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dd097-1f73-4872-86a7-c5a38849d4f0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:19.024685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:19.030513 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acce34-7202-4543-8081-f152f0720f42 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:19.071607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e66c8-7047-4047-b476-1d16cfaa8be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:19.072718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e66c8-7047-4047-b476-1d16cfaa8be2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:19.073232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e66c8-7047-4047-b476-1d16cfaa8be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:19.073737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e66c8-7047-4047-b476-1d16cfaa8be2 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:19.074763 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e66c8-7047-4047-b476-1d16cfaa8be2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:19.075329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e66c8-7047-4047-b476-1d16cfaa8be2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:19.077257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e66c8-7047-4047-b476-1d16cfaa8be2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:19.096180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 296/591] 10.222.0.86 () {56 vars in 1137 bytes} [Thu Jul 28 17:53:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:19.097506 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e66c8-7047-4047-b476-1d16cfaa8be2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:19.114731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dd097-1f73-4872-86a7-c5a38849d4f0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:19.150063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb973610-6f69-48c3-8f30-9faf583cbd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:19.150680 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb973610-6f69-48c3-8f30-9faf583cbd67 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:19.150953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb973610-6f69-48c3-8f30-9faf583cbd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:19.151227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb973610-6f69-48c3-8f30-9faf583cbd67 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:19.151992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb973610-6f69-48c3-8f30-9faf583cbd67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:19.152210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb973610-6f69-48c3-8f30-9faf583cbd67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:19.153603 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb973610-6f69-48c3-8f30-9faf583cbd67 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:19.169003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 296/592] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:19.169003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb973610-6f69-48c3-8f30-9faf583cbd67 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:19.268145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:19.284650 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb973610-6f69-48c3-8f30-9faf583cbd67 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:19.297189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e66c8-7047-4047-b476-1d16cfaa8be2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:19.325738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:19.329304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08f03b-b1d3-420b-be74-4c5f93c56722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:19.330217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08f03b-b1d3-420b-be74-4c5f93c56722 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:19.330604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08f03b-b1d3-420b-be74-4c5f93c56722 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:19.331289 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08f03b-b1d3-420b-be74-4c5f93c56722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:19.335987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162a5a4-c343-4b32-b798-17d7f6f38632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:19.336793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162a5a4-c343-4b32-b798-17d7f6f38632 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:19.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162a5a4-c343-4b32-b798-17d7f6f38632 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:19.337670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162a5a4-c343-4b32-b798-17d7f6f38632 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:19.344952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162a5a4-c343-4b32-b798-17d7f6f38632 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:19.345187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08f03b-b1d3-420b-be74-4c5f93c56722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:19.346885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08f03b-b1d3-420b-be74-4c5f93c56722 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:19.347075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162a5a4-c343-4b32-b798-17d7f6f38632 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-d162a5a4-c343-4b32-b798-17d7f6f38632 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:19.359200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:19.360306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:19.361441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:19.361441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:19.361441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:19.361441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:19.361441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:19.361441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:19.361441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:19.361441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:19.363606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 297/593] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:19.364811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162a5a4-c343-4b32-b798-17d7f6f38632 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-de08f03b-b1d3-420b-be74-4c5f93c56722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:19.373063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:19.374509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:19.375613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:19.375613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:19.375613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:19.375613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:19.375613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 297/594] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:19.375613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08f03b-b1d3-420b-be74-4c5f93c56722 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:19.510316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162a5a4-c343-4b32-b798-17d7f6f38632 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:19.510531 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08f03b-b1d3-420b-be74-4c5f93c56722 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:19.548994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bf9bc-1d4f-4d98-99ac-556331c3477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:19.549664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bf9bc-1d4f-4d98-99ac-556331c3477a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:19.550050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bf9bc-1d4f-4d98-99ac-556331c3477a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:19.550338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bf9bc-1d4f-4d98-99ac-556331c3477a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:19.554314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca9b18-a0ad-4a3e-ace6-e624be9baa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:19.555057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca9b18-a0ad-4a3e-ace6-e624be9baa84 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:19.555456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca9b18-a0ad-4a3e-ace6-e624be9baa84 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:19.555831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca9b18-a0ad-4a3e-ace6-e624be9baa84 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:19.556696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca9b18-a0ad-4a3e-ace6-e624be9baa84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:19.557137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca9b18-a0ad-4a3e-ace6-e624be9baa84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:19.558878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bf9bc-1d4f-4d98-99ac-556331c3477a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:19.560258 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca9b18-a0ad-4a3e-ace6-e624be9baa84 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:19.560371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bf9bc-1d4f-4d98-99ac-556331c3477a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:19.564545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-2f8bf9bc-1d4f-4d98-99ac-556331c3477a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:19.573155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:19.574201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:19.575140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:19.575140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:19.575140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:19.575140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:19.575140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:19.575140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:19.575140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:19.575140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:19.575140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:53:19.575140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:19.583747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 298/596] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:19.583747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bf9bc-1d4f-4d98-99ac-556331c3477a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:19.587665 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 298/596] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:19.587665 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca9b18-a0ad-4a3e-ace6-e624be9baa84 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:19.649060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:19.690612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca9b18-a0ad-4a3e-ace6-e624be9baa84 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:19.722322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bf9bc-1d4f-4d98-99ac-556331c3477a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:19.728663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4eac8-93d2-41cd-96fa-a3186c4d89db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:19.729400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4eac8-93d2-41cd-96fa-a3186c4d89db None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:19.729757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4eac8-93d2-41cd-96fa-a3186c4d89db None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:19.730109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4eac8-93d2-41cd-96fa-a3186c4d89db None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:19.731039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4eac8-93d2-41cd-96fa-a3186c4d89db None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:19.731424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4eac8-93d2-41cd-96fa-a3186c4d89db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:19.733076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4eac8-93d2-41cd-96fa-a3186c4d89db None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:19.757292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 299/597] 10.222.0.86 () {56 vars in 1139 bytes} [Thu Jul 28 17:53:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:19.758393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4eac8-93d2-41cd-96fa-a3186c4d89db None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:19.768217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f8498-b2f2-4c16-82e3-ba746c52243a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:19.768861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f8498-b2f2-4c16-82e3-ba746c52243a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:19.769175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f8498-b2f2-4c16-82e3-ba746c52243a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:19.769496 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f8498-b2f2-4c16-82e3-ba746c52243a None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:19.770205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f8498-b2f2-4c16-82e3-ba746c52243a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:19.770513 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f8498-b2f2-4c16-82e3-ba746c52243a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:19.772484 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f8498-b2f2-4c16-82e3-ba746c52243a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:19.790449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 299/598] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:19.790449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f8498-b2f2-4c16-82e3-ba746c52243a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:19.834699 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:19.886937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4eac8-93d2-41cd-96fa-a3186c4d89db None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:19.917279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f8498-b2f2-4c16-82e3-ba746c52243a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:19.924839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0943798-0425-41e3-8d32-23f3b9fc59a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:19.925666 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0943798-0425-41e3-8d32-23f3b9fc59a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:19.926077 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0943798-0425-41e3-8d32-23f3b9fc59a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:19.926459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0943798-0425-41e3-8d32-23f3b9fc59a9 None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/c47f1552e73648ceb8a421c0543c2974/roles/ce68cefcc9b14645bbb5a2b411a23a44` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:19.927224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:19.943442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0943798-0425-41e3-8d32-23f3b9fc59a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=c47f1552e73648ceb8a421c0543c2974, role_id=ce68cefcc9b14645bbb5a2b411a23a44)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:19.945187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0943798-0425-41e3-8d32-23f3b9fc59a9 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:19.960003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81ddc1-230b-491c-bf06-d6ffca86ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:19.960957 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81ddc1-230b-491c-bf06-d6ffca86ecb2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:19.961478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81ddc1-230b-491c-bf06-d6ffca86ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:19.961902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81ddc1-230b-491c-bf06-d6ffca86ecb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:19.973054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81ddc1-230b-491c-bf06-d6ffca86ecb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:19.974686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81ddc1-230b-491c-bf06-d6ffca86ecb2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ae81ddc1-230b-491c-bf06-d6ffca86ecb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:19.989950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:19.991475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:19.992588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:19.992588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:19.992588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:19.992588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:19.994151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 300/599] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:19.995091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81ddc1-230b-491c-bf06-d6ffca86ecb2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:20.029050 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 300/600] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:53:19 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/c47f1552e73648ceb8a421c0543c2974/roles/ce68cefcc9b14645bbb5a2b411a23a44 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:20.030238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0943798-0425-41e3-8d32-23f3b9fc59a9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:20.171554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81ddc1-230b-491c-bf06-d6ffca86ecb2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:20.243585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0fac-bf29-42b6-af87-3b650400f374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:20.244318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0fac-bf29-42b6-af87-3b650400f374 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.244601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0fac-bf29-42b6-af87-3b650400f374 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.244871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0fac-bf29-42b6-af87-3b650400f374 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.247505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0943798-0425-41e3-8d32-23f3b9fc59a9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:20.252721 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:20.257974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a0fac-bf29-42b6-af87-3b650400f374 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:20.259537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a0fac-bf29-42b6-af87-3b650400f374 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-263a0fac-bf29-42b6-af87-3b650400f374 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:20.273641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:20.274583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:20.275482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:20.275482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:20.275482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:20.275482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:20.275482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:20.275482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:20.275482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:20.275482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:20.275482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 301/601] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:20.281332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0fac-bf29-42b6-af87-3b650400f374 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:20.290856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88fcfe-7e48-41f6-8efa-229d1f782677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:20.291564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88fcfe-7e48-41f6-8efa-229d1f782677 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.291910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88fcfe-7e48-41f6-8efa-229d1f782677 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.292345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88fcfe-7e48-41f6-8efa-229d1f782677 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.293158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88fcfe-7e48-41f6-8efa-229d1f782677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:20.293536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88fcfe-7e48-41f6-8efa-229d1f782677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:20.294979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88fcfe-7e48-41f6-8efa-229d1f782677 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:20.308819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:20.317091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 301/602] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:20.318309 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88fcfe-7e48-41f6-8efa-229d1f782677 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:20.412753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0fac-bf29-42b6-af87-3b650400f374 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:20.444126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88fcfe-7e48-41f6-8efa-229d1f782677 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:20.450227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6427874-9a33-4f6e-98b7-f7fcd93307b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:20.450851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6427874-9a33-4f6e-98b7-f7fcd93307b0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.451128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6427874-9a33-4f6e-98b7-f7fcd93307b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.451515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6427874-9a33-4f6e-98b7-f7fcd93307b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.452300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6427874-9a33-4f6e-98b7-f7fcd93307b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:20.452605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6427874-9a33-4f6e-98b7-f7fcd93307b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:20.454067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6427874-9a33-4f6e-98b7-f7fcd93307b0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:20.467725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 302/603] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:20.469453 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbf39e-5488-41ce-89fb-ef9c0c4e8c2f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.469744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbf39e-5488-41ce-89fb-ef9c0c4e8c2f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.470022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbf39e-5488-41ce-89fb-ef9c0c4e8c2f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.471374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 303/604] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:20.474786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5d8e2-e4bd-47a2-b959-707aa158e168 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.475164 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5d8e2-e4bd-47a2-b959-707aa158e168 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.475516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5d8e2-e4bd-47a2-b959-707aa158e168 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.477271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 304/605] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:20.478240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5d8e2-e4bd-47a2-b959-707aa158e168 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:20.485382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab6f94-3cff-4d80-9492-cca78a76ba98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:20.486258 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab6f94-3cff-4d80-9492-cca78a76ba98 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.486712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab6f94-3cff-4d80-9492-cca78a76ba98 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.487185 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab6f94-3cff-4d80-9492-cca78a76ba98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.494201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab6f94-3cff-4d80-9492-cca78a76ba98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:20.495839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab6f94-3cff-4d80-9492-cca78a76ba98 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-1cab6f94-3cff-4d80-9492-cca78a76ba98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:20.508256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:20.509198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:20.510304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:20.510304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:20.510304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:20.510304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:20.510304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:20.510304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:20.510304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:20.510304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:20.512246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 302/606] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:20.513578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f50b79-4229-463b-8a5c-13ed909afc0b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.513862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f50b79-4229-463b-8a5c-13ed909afc0b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.514138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f50b79-4229-463b-8a5c-13ed909afc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.540163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-96f50b79-4229-463b-8a5c-13ed909afc0b None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:20.541717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-96f50b79-4229-463b-8a5c-13ed909afc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:20.573880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:20.580529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:20.604286 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5d8e2-e4bd-47a2-b959-707aa158e168 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:20.636985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 303/607] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:20.636985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589d657-0b12-4f22-a186-9b246bc344bf None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.636985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589d657-0b12-4f22-a186-9b246bc344bf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.636985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589d657-0b12-4f22-a186-9b246bc344bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.651145 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7c000-f691-4142-9709-2917cecebdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:20.651861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7c000-f691-4142-9709-2917cecebdd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.652153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7c000-f691-4142-9709-2917cecebdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.652433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7c000-f691-4142-9709-2917cecebdd2 None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/9dbaf1e343474322b2567e2dfda99d4e/roles/ce68cefcc9b14645bbb5a2b411a23a44` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.653567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e589d657-0b12-4f22-a186-9b246bc344bf None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:20.654770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e589d657-0b12-4f22-a186-9b246bc344bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:20.670344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7c000-f691-4142-9709-2917cecebdd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=9dbaf1e343474322b2567e2dfda99d4e, role_id=ce68cefcc9b14645bbb5a2b411a23a44)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:20.672435 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7c000-f691-4142-9709-2917cecebdd2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:20.725871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 304/608] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:20.726648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589d657-0b12-4f22-a186-9b246bc344bf None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:20.773649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 305/609] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:53:20 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/9dbaf1e343474322b2567e2dfda99d4e/roles/ce68cefcc9b14645bbb5a2b411a23a44 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:20.774360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7c000-f691-4142-9709-2917cecebdd2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:20.848517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:20.877018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589d657-0b12-4f22-a186-9b246bc344bf None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:20.910614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7c000-f691-4142-9709-2917cecebdd2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:20.929508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924f560-0028-4854-b00d-ad049896e5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:20.930130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924f560-0028-4854-b00d-ad049896e5ce None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.930413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924f560-0028-4854-b00d-ad049896e5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.930736 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924f560-0028-4854-b00d-ad049896e5ce None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.932137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924f560-0028-4854-b00d-ad049896e5ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:20.932448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924f560-0028-4854-b00d-ad049896e5ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:20.934184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924f560-0028-4854-b00d-ad049896e5ce None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:20.947034 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:20.949787 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 305/610] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:20.950614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924f560-0028-4854-b00d-ad049896e5ce None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:20.957633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23025172-b75b-455b-8dff-1715d673be9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:20.958686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23025172-b75b-455b-8dff-1715d673be9f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:20.959179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23025172-b75b-455b-8dff-1715d673be9f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:20.959795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23025172-b75b-455b-8dff-1715d673be9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:20.971729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23025172-b75b-455b-8dff-1715d673be9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-23025172-b75b-455b-8dff-1715d673be9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:20.977489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:20.978428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:20.979275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:20.981662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 306/611] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:20.982712 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23025172-b75b-455b-8dff-1715d673be9f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:21.069543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924f560-0028-4854-b00d-ad049896e5ce None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:21.097335 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23025172-b75b-455b-8dff-1715d673be9f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:21.109325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd219bf-fd36-41ee-aab4-507540df985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:21.110275 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd219bf-fd36-41ee-aab4-507540df985a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:21.110862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd219bf-fd36-41ee-aab4-507540df985a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:21.111345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd219bf-fd36-41ee-aab4-507540df985a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:21.123240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd219bf-fd36-41ee-aab4-507540df985a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-1cd219bf-fd36-41ee-aab4-507540df985a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:21.129920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:21.130895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:21.131765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:21.134596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2715096-d68b-48cc-83aa-ace7c695921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:21.134852 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 306/612] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:21.135149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2715096-d68b-48cc-83aa-ace7c695921e None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:21.135433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2715096-d68b-48cc-83aa-ace7c695921e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:21.135709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2715096-d68b-48cc-83aa-ace7c695921e None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/b038336582a0435dbd47ad2c09cd23e0/roles/ce68cefcc9b14645bbb5a2b411a23a44` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:21.136488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd219bf-fd36-41ee-aab4-507540df985a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:21.140952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:21.158345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2715096-d68b-48cc-83aa-ace7c695921e None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=b038336582a0435dbd47ad2c09cd23e0, role_id=ce68cefcc9b14645bbb5a2b411a23a44)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:21.159710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2715096-d68b-48cc-83aa-ace7c695921e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:21.231497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 307/613] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:53:20 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/b038336582a0435dbd47ad2c09cd23e0/roles/ce68cefcc9b14645bbb5a2b411a23a44 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:21.232600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2715096-d68b-48cc-83aa-ace7c695921e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:21.249446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd219bf-fd36-41ee-aab4-507540df985a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:21.294426 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550204d-a835-498c-a972-54285f3c0b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:21.295525 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550204d-a835-498c-a972-54285f3c0b15 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:21.297131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550204d-a835-498c-a972-54285f3c0b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:21.297696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550204d-a835-498c-a972-54285f3c0b15 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:21.299425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550204d-a835-498c-a972-54285f3c0b15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:21.300148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550204d-a835-498c-a972-54285f3c0b15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:21.304068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550204d-a835-498c-a972-54285f3c0b15 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:21.313368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:21.322957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 307/614] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:21.329384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550204d-a835-498c-a972-54285f3c0b15 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:21.436009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:21.454695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2715096-d68b-48cc-83aa-ace7c695921e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:21.490749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550204d-a835-498c-a972-54285f3c0b15 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:21.505850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c258a06-2061-4196-9ab6-b6a464721d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:21.506599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c258a06-2061-4196-9ab6-b6a464721d3e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:21.506892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c258a06-2061-4196-9ab6-b6a464721d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:21.507174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c258a06-2061-4196-9ab6-b6a464721d3e None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:21.508304 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c258a06-2061-4196-9ab6-b6a464721d3e None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:21.508621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c258a06-2061-4196-9ab6-b6a464721d3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:21.510318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c258a06-2061-4196-9ab6-b6a464721d3e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:21.522172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 308/615] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:21.528448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c258a06-2061-4196-9ab6-b6a464721d3e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:21.534996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292d434-1fc8-4f04-a381-de28cf37bef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:21.535856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292d434-1fc8-4f04-a381-de28cf37bef8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:21.536280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292d434-1fc8-4f04-a381-de28cf37bef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:21.536698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292d434-1fc8-4f04-a381-de28cf37bef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:21.548420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292d434-1fc8-4f04-a381-de28cf37bef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:21.550215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292d434-1fc8-4f04-a381-de28cf37bef8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-4292d434-1fc8-4f04-a381-de28cf37bef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:21.575295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:21.576448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:21.581786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:21.581786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:21.581786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:21.581786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:21.583841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 308/616] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:21.589860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292d434-1fc8-4f04-a381-de28cf37bef8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:21.650788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:21.654076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c258a06-2061-4196-9ab6-b6a464721d3e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:21.713678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c15081-6de0-44fa-bd7d-9ac20bcc213e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:21.714681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c15081-6de0-44fa-bd7d-9ac20bcc213e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:21.715154 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c15081-6de0-44fa-bd7d-9ac20bcc213e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:21.715604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c15081-6de0-44fa-bd7d-9ac20bcc213e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:21.742109 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c15081-6de0-44fa-bd7d-9ac20bcc213e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:21.743827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c15081-6de0-44fa-bd7d-9ac20bcc213e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:21.749278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292d434-1fc8-4f04-a381-de28cf37bef8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:21.770146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-49c15081-6de0-44fa-bd7d-9ac20bcc213e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:21.775784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:21.776876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:21.777855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:21.777855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:21.777855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:21.777855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:21.786024 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 309/617] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:21.791680 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145f720-2049-4224-8634-00edf001f343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:21.791836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c15081-6de0-44fa-bd7d-9ac20bcc213e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:21.792827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145f720-2049-4224-8634-00edf001f343 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:21.793354 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145f720-2049-4224-8634-00edf001f343 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:21.793866 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145f720-2049-4224-8634-00edf001f343 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:21.795107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145f720-2049-4224-8634-00edf001f343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:21.796196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145f720-2049-4224-8634-00edf001f343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:21.797817 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145f720-2049-4224-8634-00edf001f343 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:21.813003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 309/618] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:21.825899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145f720-2049-4224-8634-00edf001f343 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:21.947123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145f720-2049-4224-8634-00edf001f343 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:21.989684 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d640b42-8569-4e1d-885a-8085519e6c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:21.990739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d640b42-8569-4e1d-885a-8085519e6c13 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:21.991797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d640b42-8569-4e1d-885a-8085519e6c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:21.992345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d640b42-8569-4e1d-885a-8085519e6c13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.002854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d640b42-8569-4e1d-885a-8085519e6c13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:22.004810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d640b42-8569-4e1d-885a-8085519e6c13 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-0d640b42-8569-4e1d-885a-8085519e6c13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:22.020146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:22.021212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:22.022165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:22.022165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:22.022165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:22.022165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:22.022165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:22.022165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:22.022165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:22.022165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:22.022165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:22.022165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:22.022707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 310/619] 10.222.0.86 () {56 vars in 1093 bytes} [Thu Jul 28 17:53:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:22.030595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d640b42-8569-4e1d-885a-8085519e6c13 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:22.053740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c15081-6de0-44fa-bd7d-9ac20bcc213e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:22.106240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:22.127297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3106118-63d1-482d-95fd-7271dc5ec8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:22.127944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3106118-63d1-482d-95fd-7271dc5ec8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.128209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3106118-63d1-482d-95fd-7271dc5ec8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.128486 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3106118-63d1-482d-95fd-7271dc5ec8d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.129224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3106118-63d1-482d-95fd-7271dc5ec8d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:22.129532 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3106118-63d1-482d-95fd-7271dc5ec8d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:22.130915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3106118-63d1-482d-95fd-7271dc5ec8d8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:22.149389 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 310/620] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:22.156037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3106118-63d1-482d-95fd-7271dc5ec8d8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:22.180693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:22.223333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d640b42-8569-4e1d-885a-8085519e6c13 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:22.259312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e089f0d-baa3-4c3c-b942-878b524fcbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:22.259861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089f0d-baa3-4c3c-b942-878b524fcbfd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.260156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089f0d-baa3-4c3c-b942-878b524fcbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.260479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089f0d-baa3-4c3c-b942-878b524fcbfd None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.261332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089f0d-baa3-4c3c-b942-878b524fcbfd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:22.261643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089f0d-baa3-4c3c-b942-878b524fcbfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:22.263141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089f0d-baa3-4c3c-b942-878b524fcbfd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:22.281890 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 311/621] 10.222.0.86 () {56 vars in 1133 bytes} [Thu Jul 28 17:53:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:22.283207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc3353-afd0-4a49-8492-5d179744bd76 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.283502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc3353-afd0-4a49-8492-5d179744bd76 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.283783 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc3353-afd0-4a49-8492-5d179744bd76 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.285254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 312/622] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:22.287373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc3353-afd0-4a49-8492-5d179744bd76 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:22.294937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3106118-63d1-482d-95fd-7271dc5ec8d8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:22.334200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f42298-6bc1-46ba-8b27-36f20e2d4b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:22.334850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f42298-6bc1-46ba-8b27-36f20e2d4b37 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.335128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f42298-6bc1-46ba-8b27-36f20e2d4b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.335408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f42298-6bc1-46ba-8b27-36f20e2d4b37 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.343838 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f42298-6bc1-46ba-8b27-36f20e2d4b37 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:22.345546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f42298-6bc1-46ba-8b27-36f20e2d4b37 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-d9f42298-6bc1-46ba-8b27-36f20e2d4b37 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:22.361902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:22.362903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:22.363847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:22.363847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:22.363847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:22.363847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:22.363847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:22.363847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:22.363847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:22.363847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:22.363847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:22.363847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:22.369398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 311/623] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:22.370702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832f8d1-bef6-4300-b86f-becb9104cc99 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.371449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832f8d1-bef6-4300-b86f-becb9104cc99 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.371727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832f8d1-bef6-4300-b86f-becb9104cc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.391474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:22.395602 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e832f8d1-bef6-4300-b86f-becb9104cc99 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:22.396817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e832f8d1-bef6-4300-b86f-becb9104cc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:22.404251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc3353-afd0-4a49-8492-5d179744bd76 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:22.448020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44ac80-0dd0-4ff8-9386-d0197d344998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:22.448711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44ac80-0dd0-4ff8-9386-d0197d344998 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.449033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44ac80-0dd0-4ff8-9386-d0197d344998 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.449360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44ac80-0dd0-4ff8-9386-d0197d344998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.513091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 312/624] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:22.513091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832f8d1-bef6-4300-b86f-becb9104cc99 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:22.513091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:22.540930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44ac80-0dd0-4ff8-9386-d0197d344998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:22.542560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44ac80-0dd0-4ff8-9386-d0197d344998 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:22.651389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:22.656114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832f8d1-bef6-4300-b86f-becb9104cc99 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:22.665424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 313/625] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:22.666865 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44ac80-0dd0-4ff8-9386-d0197d344998 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:22.701591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b192d-cfc2-4f9d-9ec5-6c61ab10a508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:22.703117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b192d-cfc2-4f9d-9ec5-6c61ab10a508 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.703702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b192d-cfc2-4f9d-9ec5-6c61ab10a508 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.704152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b192d-cfc2-4f9d-9ec5-6c61ab10a508 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.705887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b192d-cfc2-4f9d-9ec5-6c61ab10a508 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:22.706207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b192d-cfc2-4f9d-9ec5-6c61ab10a508 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:22.707684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b192d-cfc2-4f9d-9ec5-6c61ab10a508 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:22.731603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 313/626] 10.222.0.86 () {56 vars in 1135 bytes} [Thu Jul 28 17:53:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:22.731603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b192d-cfc2-4f9d-9ec5-6c61ab10a508 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:22.796785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44ac80-0dd0-4ff8-9386-d0197d344998 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:22.810516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:22.833076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a762e-ac8e-49fa-8261-505131deee4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:22.833730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a762e-ac8e-49fa-8261-505131deee4c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.834066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a762e-ac8e-49fa-8261-505131deee4c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.834345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a762e-ac8e-49fa-8261-505131deee4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.846075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a762e-ac8e-49fa-8261-505131deee4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-249a762e-ac8e-49fa-8261-505131deee4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:22.851601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:22.852579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:22.853644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:22.857124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 314/627] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:22.857932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a762e-ac8e-49fa-8261-505131deee4c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:22.867362 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b192d-cfc2-4f9d-9ec5-6c61ab10a508 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:22.921292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082c0d5-83b0-4902-ac01-01cc4620c480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:22.922319 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082c0d5-83b0-4902-ac01-01cc4620c480 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.923354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082c0d5-83b0-4902-ac01-01cc4620c480 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.924027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082c0d5-83b0-4902-ac01-01cc4620c480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.928085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:22.939866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082c0d5-83b0-4902-ac01-01cc4620c480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:22.941563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082c0d5-83b0-4902-ac01-01cc4620c480 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b082c0d5-83b0-4902-ac01-01cc4620c480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:22.959768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:22.960726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:22.961779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:22.961779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:22.961779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:22.961779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:22.961779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 314/628] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:22.962524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082c0d5-83b0-4902-ac01-01cc4620c480 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:22.988735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a762e-ac8e-49fa-8261-505131deee4c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.030182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3409c-545f-4fbb-adea-8f3c8c04dbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.030826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3409c-545f-4fbb-adea-8f3c8c04dbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.031112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3409c-545f-4fbb-adea-8f3c8c04dbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.031394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3409c-545f-4fbb-adea-8f3c8c04dbf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.042259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3409c-545f-4fbb-adea-8f3c8c04dbf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.043715 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3409c-545f-4fbb-adea-8f3c8c04dbf0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-03b3409c-545f-4fbb-adea-8f3c8c04dbf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:23.062579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:23.063466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:23.064406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:23.064406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:23.064406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:23.064406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:23.065286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 315/629] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:23.066547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfc77a-c34b-4b11-b0f1-17d3bf964b19 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.066845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfc77a-c34b-4b11-b0f1-17d3bf964b19 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.067114 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfc77a-c34b-4b11-b0f1-17d3bf964b19 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.072689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 316/630] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:23.072689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfc77a-c34b-4b11-b0f1-17d3bf964b19 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.072897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082c0d5-83b0-4902-ac01-01cc4620c480 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.085395 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:23.108147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed85a5-b672-4ca4-9e75-21c0f87fd06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.108853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed85a5-b672-4ca4-9e75-21c0f87fd06f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.109330 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed85a5-b672-4ca4-9e75-21c0f87fd06f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.109588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed85a5-b672-4ca4-9e75-21c0f87fd06f None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.110391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed85a5-b672-4ca4-9e75-21c0f87fd06f None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:23.110762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed85a5-b672-4ca4-9e75-21c0f87fd06f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.112201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed85a5-b672-4ca4-9e75-21c0f87fd06f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.127122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 315/631] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:23.127122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed85a5-b672-4ca4-9e75-21c0f87fd06f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.186850 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfc77a-c34b-4b11-b0f1-17d3bf964b19 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.196431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:23.223778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643116fb-b5e2-40cd-af2f-206fc8f7e86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.224537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643116fb-b5e2-40cd-af2f-206fc8f7e86a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.225081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643116fb-b5e2-40cd-af2f-206fc8f7e86a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.225762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643116fb-b5e2-40cd-af2f-206fc8f7e86a None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.226846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643116fb-b5e2-40cd-af2f-206fc8f7e86a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:23.227257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643116fb-b5e2-40cd-af2f-206fc8f7e86a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.229502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643116fb-b5e2-40cd-af2f-206fc8f7e86a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.231362 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed85a5-b672-4ca4-9e75-21c0f87fd06f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.249807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 317/632] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:23.249807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecccd8-b3ee-4fa6-9dd3-b9b2f6345e1c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.249807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecccd8-b3ee-4fa6-9dd3-b9b2f6345e1c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.249807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecccd8-b3ee-4fa6-9dd3-b9b2f6345e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.270799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1eecccd8-b3ee-4fa6-9dd3-b9b2f6345e1c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:23.272145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1eecccd8-b3ee-4fa6-9dd3-b9b2f6345e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:23.274653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d566096-1a5f-4263-9050-0d69b983e160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.275564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d566096-1a5f-4263-9050-0d69b983e160 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.275985 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d566096-1a5f-4263-9050-0d69b983e160 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.276390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d566096-1a5f-4263-9050-0d69b983e160 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.277471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d566096-1a5f-4263-9050-0d69b983e160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:23.277935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d566096-1a5f-4263-9050-0d69b983e160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.280222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d566096-1a5f-4263-9050-0d69b983e160 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.299727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 316/633] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:23.301432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d566096-1a5f-4263-9050-0d69b983e160 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.344796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:23.361671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 318/634] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:23.365961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecccd8-b3ee-4fa6-9dd3-b9b2f6345e1c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.433074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d566096-1a5f-4263-9050-0d69b983e160 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.468015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c419a4-203d-4f68-b4b8-17192e24fb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.468668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c419a4-203d-4f68-b4b8-17192e24fb35 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.468950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c419a4-203d-4f68-b4b8-17192e24fb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.469333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c419a4-203d-4f68-b4b8-17192e24fb35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.473871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:23.477803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecccd8-b3ee-4fa6-9dd3-b9b2f6345e1c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.480487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c419a4-203d-4f68-b4b8-17192e24fb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.481963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c419a4-203d-4f68-b4b8-17192e24fb35 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-e0c419a4-203d-4f68-b4b8-17192e24fb35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:23.497245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:23.498428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:23.500204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:23.500204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:23.500204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:23.500204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:23.504037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 317/635] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:23.504612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c419a4-203d-4f68-b4b8-17192e24fb35 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.516484 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12b0c3-da1f-4611-b056-5920b4bdf1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.517301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12b0c3-da1f-4611-b056-5920b4bdf1de None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.517676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12b0c3-da1f-4611-b056-5920b4bdf1de None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.518053 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12b0c3-da1f-4611-b056-5920b4bdf1de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.524752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12b0c3-da1f-4611-b056-5920b4bdf1de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.526494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12b0c3-da1f-4611-b056-5920b4bdf1de None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-8f12b0c3-da1f-4611-b056-5920b4bdf1de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:23.542860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:23.543976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:23.545006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:23.545006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:23.545006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:23.545006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:23.545006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:23.545006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:23.545006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:23.545006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:23.546200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 319/636] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:23.551535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12b0c3-da1f-4611-b056-5920b4bdf1de None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.595184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:23.615030 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c419a4-203d-4f68-b4b8-17192e24fb35 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.657592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bb5fd-6ebc-495c-bf9d-67d403581e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.658259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bb5fd-6ebc-495c-bf9d-67d403581e12 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.658546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bb5fd-6ebc-495c-bf9d-67d403581e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.658823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bb5fd-6ebc-495c-bf9d-67d403581e12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.667556 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05bb5fd-6ebc-495c-bf9d-67d403581e12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.670343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05bb5fd-6ebc-495c-bf9d-67d403581e12 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.676059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12b0c3-da1f-4611-b056-5920b4bdf1de None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c05bb5fd-6ebc-495c-bf9d-67d403581e12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:23.685970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:23.686904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:23.687917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:23.687917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:23.687917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:23.687917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:23.687917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:23.687917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:23.687917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:23.687917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:23.687917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 318/637] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:23.689156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bb5fd-6ebc-495c-bf9d-67d403581e12 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.715784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd187cdc-112e-4f78-804d-a1f575eef609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.716415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd187cdc-112e-4f78-804d-a1f575eef609 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.716691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd187cdc-112e-4f78-804d-a1f575eef609 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.716981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd187cdc-112e-4f78-804d-a1f575eef609 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.727179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd187cdc-112e-4f78-804d-a1f575eef609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-bd187cdc-112e-4f78-804d-a1f575eef609 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:23.731992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:23.733161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:23.734546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 320/638] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:23.735244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd187cdc-112e-4f78-804d-a1f575eef609 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.753240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:23.794327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bb5fd-6ebc-495c-bf9d-67d403581e12 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.826962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e771051-52f5-420d-8bcb-85060ad7eb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.827577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e771051-52f5-420d-8bcb-85060ad7eb43 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.827864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e771051-52f5-420d-8bcb-85060ad7eb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.828140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e771051-52f5-420d-8bcb-85060ad7eb43 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.828829 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e771051-52f5-420d-8bcb-85060ad7eb43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:23.829187 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e771051-52f5-420d-8bcb-85060ad7eb43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.830567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e771051-52f5-420d-8bcb-85060ad7eb43 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.832595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd187cdc-112e-4f78-804d-a1f575eef609 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.842168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 319/639] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:23.843460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e771051-52f5-420d-8bcb-85060ad7eb43 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.852162 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:23.867849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57acd4-8b5a-419a-9a55-5c25e2b83473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.868469 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57acd4-8b5a-419a-9a55-5c25e2b83473 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.868757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57acd4-8b5a-419a-9a55-5c25e2b83473 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.869066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57acd4-8b5a-419a-9a55-5c25e2b83473 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.869835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57acd4-8b5a-419a-9a55-5c25e2b83473 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:23.870145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57acd4-8b5a-419a-9a55-5c25e2b83473 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.871562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57acd4-8b5a-419a-9a55-5c25e2b83473 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.883771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 321/640] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:23.885089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2335914-5ddf-47fc-ba55-c92b5983fe9f None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.885382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2335914-5ddf-47fc-ba55-c92b5983fe9f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.885659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2335914-5ddf-47fc-ba55-c92b5983fe9f None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.887019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 322/641] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:23.887624 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2335914-5ddf-47fc-ba55-c92b5983fe9f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.944941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e771051-52f5-420d-8bcb-85060ad7eb43 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.966863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:23.978544 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbccaa4-733e-4fe4-9440-d18cf53ce737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.979193 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbccaa4-733e-4fe4-9440-d18cf53ce737 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.979476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbccaa4-733e-4fe4-9440-d18cf53ce737 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.979749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbccaa4-733e-4fe4-9440-d18cf53ce737 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.980501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbccaa4-733e-4fe4-9440-d18cf53ce737 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:23.980807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbccaa4-733e-4fe4-9440-d18cf53ce737 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.982254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbccaa4-733e-4fe4-9440-d18cf53ce737 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.988626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2335914-5ddf-47fc-ba55-c92b5983fe9f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.994255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 320/642] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:23.994857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbccaa4-733e-4fe4-9440-d18cf53ce737 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.023334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc82f19-2129-4fc7-9aab-f3cf543cbb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.023950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc82f19-2129-4fc7-9aab-f3cf543cbb67 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.024235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc82f19-2129-4fc7-9aab-f3cf543cbb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.024512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc82f19-2129-4fc7-9aab-f3cf543cbb67 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.025351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc82f19-2129-4fc7-9aab-f3cf543cbb67 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:24.025655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc82f19-2129-4fc7-9aab-f3cf543cbb67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.027026 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc82f19-2129-4fc7-9aab-f3cf543cbb67 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.034849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 323/643] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:24.035632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc82f19-2129-4fc7-9aab-f3cf543cbb67 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.063625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:24.098424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbccaa4-733e-4fe4-9440-d18cf53ce737 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.132792 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb04c3d-ea00-47a8-a2d8-7cc0e19878d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.133403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb04c3d-ea00-47a8-a2d8-7cc0e19878d9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.133687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb04c3d-ea00-47a8-a2d8-7cc0e19878d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.133958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb04c3d-ea00-47a8-a2d8-7cc0e19878d9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.135479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc82f19-2129-4fc7-9aab-f3cf543cbb67 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.141864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb04c3d-ea00-47a8-a2d8-7cc0e19878d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.143295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb04c3d-ea00-47a8-a2d8-7cc0e19878d9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-adb04c3d-ea00-47a8-a2d8-7cc0e19878d9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:24.154291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:24.155860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:24.157001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:24.157001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:24.157001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:24.157001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:24.157001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:24.157001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:24.157001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:24.157001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:24.157001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:53:24.157001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:24.157001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 321/644] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:24.157638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47b77-f31b-4a96-8715-3eb88caaebbe None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.157638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47b77-f31b-4a96-8715-3eb88caaebbe None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.157770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47b77-f31b-4a96-8715-3eb88caaebbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.173355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deefb7d-7672-4495-8dd1-2a53fe2550e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.173979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefb7d-7672-4495-8dd1-2a53fe2550e7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.174265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefb7d-7672-4495-8dd1-2a53fe2550e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.174543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefb7d-7672-4495-8dd1-2a53fe2550e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.174762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-94e47b77-f31b-4a96-8715-3eb88caaebbe None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:24.175361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deefb7d-7672-4495-8dd1-2a53fe2550e7 None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=7d7bcb4e7c434622b650cdfc500a9680) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:24.175671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deefb7d-7672-4495-8dd1-2a53fe2550e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.175997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-94e47b77-f31b-4a96-8715-3eb88caaebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:24.177146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deefb7d-7672-4495-8dd1-2a53fe2550e7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.183912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 324/645] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:24 2022] GET /identity/v3/role_assignments?user.id=7d7bcb4e7c434622b650cdfc500a9680&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:24.184467 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deefb7d-7672-4495-8dd1-2a53fe2550e7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.203154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:24.260991 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 322/646] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:24.260991 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e47b77-f31b-4a96-8715-3eb88caaebbe None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.331076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deefb7d-7672-4495-8dd1-2a53fe2550e7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.332011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:24.373788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf29620-f47a-4f0e-b732-7a0a4408d563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.374466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf29620-f47a-4f0e-b732-7a0a4408d563 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.374737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf29620-f47a-4f0e-b732-7a0a4408d563 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.375013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf29620-f47a-4f0e-b732-7a0a4408d563 None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/3077d844257c48b3b4ac2db9177fb6df/roles/ce68cefcc9b14645bbb5a2b411a23a44` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.384572 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e47b77-f31b-4a96-8715-3eb88caaebbe None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.391807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf29620-f47a-4f0e-b732-7a0a4408d563 None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=3077d844257c48b3b4ac2db9177fb6df, role_id=ce68cefcc9b14645bbb5a2b411a23a44)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.393532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf29620-f47a-4f0e-b732-7a0a4408d563 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.428571 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87930d13-b268-4a4d-9c25-6ff5e09e9c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.429370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930d13-b268-4a4d-9c25-6ff5e09e9c33 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.429652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930d13-b268-4a4d-9c25-6ff5e09e9c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.429928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930d13-b268-4a4d-9c25-6ff5e09e9c33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.440952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87930d13-b268-4a4d-9c25-6ff5e09e9c33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.442429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87930d13-b268-4a4d-9c25-6ff5e09e9c33 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-87930d13-b268-4a4d-9c25-6ff5e09e9c33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:24.457716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:24.459401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:24.460695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:24.460695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:24.460695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:24.460695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:24.460695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 323/647] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:24.460695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87930d13-b268-4a4d-9c25-6ff5e09e9c33 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.476542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 325/648] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:53:24 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/3077d844257c48b3b4ac2db9177fb6df/roles/ce68cefcc9b14645bbb5a2b411a23a44 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:24.477965 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf29620-f47a-4f0e-b732-7a0a4408d563 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.565310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:24.598506 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87930d13-b268-4a4d-9c25-6ff5e09e9c33 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.611248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:24.634957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf29620-f47a-4f0e-b732-7a0a4408d563 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.649692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a741a-cdd0-434c-85e4-093a595a9f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.650338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a741a-cdd0-434c-85e4-093a595a9f69 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.650628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a741a-cdd0-434c-85e4-093a595a9f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.650906 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a741a-cdd0-434c-85e4-093a595a9f69 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.651806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a741a-cdd0-434c-85e4-093a595a9f69 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:24.652058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a741a-cdd0-434c-85e4-093a595a9f69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.653674 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a741a-cdd0-434c-85e4-093a595a9f69 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.675273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b05e6-cf8a-4a0f-bc03-5d94ab6deb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.675920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b05e6-cf8a-4a0f-bc03-5d94ab6deb78 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.676204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b05e6-cf8a-4a0f-bc03-5d94ab6deb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.676483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b05e6-cf8a-4a0f-bc03-5d94ab6deb78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.679561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 324/649] 10.222.0.86 () {56 vars in 1137 bytes} [Thu Jul 28 17:53:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:24.680366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a741a-cdd0-434c-85e4-093a595a9f69 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.688761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b05e6-cf8a-4a0f-bc03-5d94ab6deb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-5e2b05e6-cf8a-4a0f-bc03-5d94ab6deb78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:24.695293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:24.696230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:24.697633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 326/650] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:24.698572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b05e6-cf8a-4a0f-bc03-5d94ab6deb78 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.791883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a741a-cdd0-434c-85e4-093a595a9f69 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.822081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b05e6-cf8a-4a0f-bc03-5d94ab6deb78 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.830463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd031e13-a16c-421d-b7e3-ff42b59d0ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.831412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd031e13-a16c-421d-b7e3-ff42b59d0ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.831923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd031e13-a16c-421d-b7e3-ff42b59d0ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.832457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd031e13-a16c-421d-b7e3-ff42b59d0ee7 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.833411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd031e13-a16c-421d-b7e3-ff42b59d0ee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:24.833923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd031e13-a16c-421d-b7e3-ff42b59d0ee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.835490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd031e13-a16c-421d-b7e3-ff42b59d0ee7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.837374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:24.853369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 325/651] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:24.854540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd031e13-a16c-421d-b7e3-ff42b59d0ee7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.860002 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32524dfd-1c25-4739-a7e0-415efd9cef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.860002 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32524dfd-1c25-4739-a7e0-415efd9cef73 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.860260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32524dfd-1c25-4739-a7e0-415efd9cef73 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.860445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32524dfd-1c25-4739-a7e0-415efd9cef73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.917472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:24.933133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32524dfd-1c25-4739-a7e0-415efd9cef73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.934606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32524dfd-1c25-4739-a7e0-415efd9cef73 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.964239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd031e13-a16c-421d-b7e3-ff42b59d0ee7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.001290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d1bf5-ed30-49e4-82ac-ace05c22c0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.002575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d1bf5-ed30-49e4-82ac-ace05c22c0aa None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.003063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d1bf5-ed30-49e4-82ac-ace05c22c0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.003537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d1bf5-ed30-49e4-82ac-ace05c22c0aa None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.004460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d1bf5-ed30-49e4-82ac-ace05c22c0aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:25.005026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d1bf5-ed30-49e4-82ac-ace05c22c0aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.006596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d1bf5-ed30-49e4-82ac-ace05c22c0aa None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.016089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 326/652] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:25.017283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d1bf5-ed30-49e4-82ac-ace05c22c0aa None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.047399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 327/653] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:25.048388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32524dfd-1c25-4739-a7e0-415efd9cef73 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.063978 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:25.121732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:25.139768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d1bf5-ed30-49e4-82ac-ace05c22c0aa None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.164923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32524dfd-1c25-4739-a7e0-415efd9cef73 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.177551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934b1e4-cedd-4d49-83fd-77c88c73ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.178135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b1e4-cedd-4d49-83fd-77c88c73ea70 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.178422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b1e4-cedd-4d49-83fd-77c88c73ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.178699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b1e4-cedd-4d49-83fd-77c88c73ea70 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.187867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934b1e4-cedd-4d49-83fd-77c88c73ea70 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.189378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934b1e4-cedd-4d49-83fd-77c88c73ea70 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.201647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513329a-9212-4697-a53d-951c1ee9a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.202939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513329a-9212-4697-a53d-951c1ee9a391 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-5934b1e4-cedd-4d49-83fd-77c88c73ea70 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:25.203264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:25.204260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:25.205824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:25.205824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:25.205824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:25.205824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:25.205824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:25.205824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:25.205824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:25.205824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:25.205824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:53:25.205824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:25.207082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513329a-9212-4697-a53d-951c1ee9a391 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.207082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513329a-9212-4697-a53d-951c1ee9a391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.211011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 327/654] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:25.211910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934b1e4-cedd-4d49-83fd-77c88c73ea70 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.219383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513329a-9212-4697-a53d-951c1ee9a391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.220957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513329a-9212-4697-a53d-951c1ee9a391 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-6513329a-9212-4697-a53d-951c1ee9a391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:25.237121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:25.239039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:25.241325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:25.241325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:25.241325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:25.241325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:25.245311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 328/655] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:25.245311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513329a-9212-4697-a53d-951c1ee9a391 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.315399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:25.323933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934b1e4-cedd-4d49-83fd-77c88c73ea70 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.352022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513329a-9212-4697-a53d-951c1ee9a391 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.367377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3c0a1-0a7f-411f-817e-c95c1c122114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.368747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3c0a1-0a7f-411f-817e-c95c1c122114 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.369305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3c0a1-0a7f-411f-817e-c95c1c122114 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.370085 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3c0a1-0a7f-411f-817e-c95c1c122114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.384464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3c0a1-0a7f-411f-817e-c95c1c122114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.386239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3c0a1-0a7f-411f-817e-c95c1c122114 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.393438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843fac7-8e51-49ba-8030-01e247fe9bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.394315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843fac7-8e51-49ba-8030-01e247fe9bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.394797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843fac7-8e51-49ba-8030-01e247fe9bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.395073 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843fac7-8e51-49ba-8030-01e247fe9bb6 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.395890 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843fac7-8e51-49ba-8030-01e247fe9bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:25.396203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843fac7-8e51-49ba-8030-01e247fe9bb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.397814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843fac7-8e51-49ba-8030-01e247fe9bb6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.398641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-1ca3c0a1-0a7f-411f-817e-c95c1c122114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:25.409667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:25.410596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:25.411493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:25.411493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:25.411493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:25.411493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:25.415997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 328/656] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:25.423642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3c0a1-0a7f-411f-817e-c95c1c122114 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.426897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 329/657] 10.222.0.86 () {56 vars in 1137 bytes} [Thu Jul 28 17:53:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:25.427984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843fac7-8e51-49ba-8030-01e247fe9bb6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.529470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3c0a1-0a7f-411f-817e-c95c1c122114 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.540950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843fac7-8e51-49ba-8030-01e247fe9bb6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.565097 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a36b6-38f9-478e-936d-ec7f035811ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.565724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a36b6-38f9-478e-936d-ec7f035811ed None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.566036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a36b6-38f9-478e-936d-ec7f035811ed None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.566318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a36b6-38f9-478e-936d-ec7f035811ed None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.566986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a36b6-38f9-478e-936d-ec7f035811ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:25.567292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a36b6-38f9-478e-936d-ec7f035811ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.568673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a36b6-38f9-478e-936d-ec7f035811ed None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.569703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:25.585691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 329/658] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:25.585691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a36b6-38f9-478e-936d-ec7f035811ed None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.585941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5fde6-6f71-42f0-aeb8-184a52c6f324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.585941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5fde6-6f71-42f0-aeb8-184a52c6f324 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.586567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5fde6-6f71-42f0-aeb8-184a52c6f324 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.586567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5fde6-6f71-42f0-aeb8-184a52c6f324 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.587558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5fde6-6f71-42f0-aeb8-184a52c6f324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:25.587871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5fde6-6f71-42f0-aeb8-184a52c6f324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.589513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5fde6-6f71-42f0-aeb8-184a52c6f324 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.608880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 330/659] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:25.610296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5fde6-6f71-42f0-aeb8-184a52c6f324 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.666381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:25.719720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a36b6-38f9-478e-936d-ec7f035811ed None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.757932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5fde6-6f71-42f0-aeb8-184a52c6f324 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.758686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7b225-69d7-487f-b8c2-15211246d426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.759583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7b225-69d7-487f-b8c2-15211246d426 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.759927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7b225-69d7-487f-b8c2-15211246d426 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.760270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7b225-69d7-487f-b8c2-15211246d426 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.802909 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fd8d3-115b-43c4-b19a-f0904ae9014d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.804000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fd8d3-115b-43c4-b19a-f0904ae9014d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.804500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fd8d3-115b-43c4-b19a-f0904ae9014d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.805069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fd8d3-115b-43c4-b19a-f0904ae9014d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.814005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fd8d3-115b-43c4-b19a-f0904ae9014d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.815723 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fd8d3-115b-43c4-b19a-f0904ae9014d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-387fd8d3-115b-43c4-b19a-f0904ae9014d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:25.827701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:25.828725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:25.829712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:25.829712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:25.829712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:25.829712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:25.829712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:25.829712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:25.829712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:25.829712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:25.829712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:53:25.829712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:25.836380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7b225-69d7-487f-b8c2-15211246d426 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.836380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7b225-69d7-487f-b8c2-15211246d426 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.837005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 331/660] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:25.838132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fd8d3-115b-43c4-b19a-f0904ae9014d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.887807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:25.898245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:25.946589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 330/661] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:25.947324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fd8d3-115b-43c4-b19a-f0904ae9014d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.947463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7b225-69d7-487f-b8c2-15211246d426 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.985070 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d9286-afb0-4373-ad64-e03d43e48492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.986038 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d9286-afb0-4373-ad64-e03d43e48492 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.986565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d9286-afb0-4373-ad64-e03d43e48492 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.987023 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d9286-afb0-4373-ad64-e03d43e48492 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.994392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d9286-afb0-4373-ad64-e03d43e48492 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.996109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d9286-afb0-4373-ad64-e03d43e48492 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-6f7d9286-afb0-4373-ad64-e03d43e48492 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:26.009557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:26.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:26.011480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:26.011480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:26.011480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:26.011480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:26.011480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:26.011480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:26.011480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:26.011480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:26.013988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 332/662] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:26.015003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d9286-afb0-4373-ad64-e03d43e48492 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.059595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7b225-69d7-487f-b8c2-15211246d426 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.099470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333ce17-cfb3-483c-95f9-53c76f8b8ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.100135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333ce17-cfb3-483c-95f9-53c76f8b8ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.100421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333ce17-cfb3-483c-95f9-53c76f8b8ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.100702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333ce17-cfb3-483c-95f9-53c76f8b8ea7 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.102892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333ce17-cfb3-483c-95f9-53c76f8b8ea7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:26.102892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333ce17-cfb3-483c-95f9-53c76f8b8ea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.103273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333ce17-cfb3-483c-95f9-53c76f8b8ea7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.132879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 331/663] 10.222.0.86 () {56 vars in 1139 bytes} [Thu Jul 28 17:53:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:26.132879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333ce17-cfb3-483c-95f9-53c76f8b8ea7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.160753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:26.162341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d9286-afb0-4373-ad64-e03d43e48492 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.168172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:26.203286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759428cc-08d2-4ab6-a5a1-200b475dcc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.203942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759428cc-08d2-4ab6-a5a1-200b475dcc73 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.204233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759428cc-08d2-4ab6-a5a1-200b475dcc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.204574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759428cc-08d2-4ab6-a5a1-200b475dcc73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.216245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759428cc-08d2-4ab6-a5a1-200b475dcc73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.218355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759428cc-08d2-4ab6-a5a1-200b475dcc73 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-759428cc-08d2-4ab6-a5a1-200b475dcc73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:26.235737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:26.236673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:26.238014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:26.238014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:26.238014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:26.238014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:26.241506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 333/664] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:26.242390 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759428cc-08d2-4ab6-a5a1-200b475dcc73 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.273086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333ce17-cfb3-483c-95f9-53c76f8b8ea7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.311263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cbb31-25ad-47ed-b550-2b7fb11e585d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.312035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cbb31-25ad-47ed-b550-2b7fb11e585d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.312382 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cbb31-25ad-47ed-b550-2b7fb11e585d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.312727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cbb31-25ad-47ed-b550-2b7fb11e585d None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.313596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cbb31-25ad-47ed-b550-2b7fb11e585d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:26.313975 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cbb31-25ad-47ed-b550-2b7fb11e585d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.315472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cbb31-25ad-47ed-b550-2b7fb11e585d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.328163 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 332/665] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:26.329404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cbb31-25ad-47ed-b550-2b7fb11e585d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.354881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759428cc-08d2-4ab6-a5a1-200b475dcc73 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.389776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2172b-c3c2-4c19-9f7b-10f68589f60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.390403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2172b-c3c2-4c19-9f7b-10f68589f60e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.390687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2172b-c3c2-4c19-9f7b-10f68589f60e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.391030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2172b-c3c2-4c19-9f7b-10f68589f60e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.418591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:26.423440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:26.446374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cbb31-25ad-47ed-b550-2b7fb11e585d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.480901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2172b-c3c2-4c19-9f7b-10f68589f60e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.482661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2172b-c3c2-4c19-9f7b-10f68589f60e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.492013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a8c27-321f-45ad-886b-bcb5aa896cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.492574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8c27-321f-45ad-886b-bcb5aa896cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.492848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8c27-321f-45ad-886b-bcb5aa896cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.493153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8c27-321f-45ad-886b-bcb5aa896cc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.493846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a8c27-321f-45ad-886b-bcb5aa896cc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:26.494362 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a8c27-321f-45ad-886b-bcb5aa896cc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.495692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a8c27-321f-45ad-886b-bcb5aa896cc3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.508575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 333/666] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:26.513168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724c1eb-ff97-4711-98e4-f8b5a55577a3 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.513168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724c1eb-ff97-4711-98e4-f8b5a55577a3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.513168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724c1eb-ff97-4711-98e4-f8b5a55577a3 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.516096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 334/667] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:26.516842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724c1eb-ff97-4711-98e4-f8b5a55577a3 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.596687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 334/668] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:26.596687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2172b-c3c2-4c19-9f7b-10f68589f60e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.622441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724c1eb-ff97-4711-98e4-f8b5a55577a3 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.652146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:26.658861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229ae6a-a129-428b-b920-57557d7e9109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.659561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229ae6a-a129-428b-b920-57557d7e9109 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.659920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229ae6a-a129-428b-b920-57557d7e9109 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.660272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229ae6a-a129-428b-b920-57557d7e9109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.661173 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229ae6a-a129-428b-b920-57557d7e9109 None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=c47f1552e73648ceb8a421c0543c2974) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:26.661560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229ae6a-a129-428b-b920-57557d7e9109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.663054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229ae6a-a129-428b-b920-57557d7e9109 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.668841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 335/669] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/role_assignments?user.id=c47f1552e73648ceb8a421c0543c2974&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:26.670023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bf0c2-3ecb-4d7d-807e-7bb71befd788 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.670314 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bf0c2-3ecb-4d7d-807e-7bb71befd788 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.670882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bf0c2-3ecb-4d7d-807e-7bb71befd788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.677068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:26.697405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-426bf0c2-3ecb-4d7d-807e-7bb71befd788 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:26.698741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2172b-c3c2-4c19-9f7b-10f68589f60e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.699224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-426bf0c2-3ecb-4d7d-807e-7bb71befd788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:26.740888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a93f3-2c28-44a4-be7c-9017e13b839f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.741587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a93f3-2c28-44a4-be7c-9017e13b839f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.741876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a93f3-2c28-44a4-be7c-9017e13b839f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.742212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a93f3-2c28-44a4-be7c-9017e13b839f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.751730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a93f3-2c28-44a4-be7c-9017e13b839f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.753271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a93f3-2c28-44a4-be7c-9017e13b839f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-3f5a93f3-2c28-44a4-be7c-9017e13b839f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:26.766348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:26.767496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:26.768585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:26.768585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:26.768585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:26.768585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:26.768585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:26.768585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:26.768585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:26.768585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:26.772518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 335/670] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:26.773539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a93f3-2c28-44a4-be7c-9017e13b839f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.787573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 336/671] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:26.787573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac03aec-269c-479b-9778-ea6662807f5e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.787573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac03aec-269c-479b-9778-ea6662807f5e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.787573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac03aec-269c-479b-9778-ea6662807f5e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.787573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 337/672] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:26.787573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac03aec-269c-479b-9778-ea6662807f5e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.902521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a93f3-2c28-44a4-be7c-9017e13b839f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.921483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac03aec-269c-479b-9778-ea6662807f5e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.923184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:26.942227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd56fc-ccfd-4c42-b7bd-252db3e9ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.942862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd56fc-ccfd-4c42-b7bd-252db3e9ea4f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.943142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd56fc-ccfd-4c42-b7bd-252db3e9ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.943417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd56fc-ccfd-4c42-b7bd-252db3e9ea4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.954549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:26.955401 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd56fc-ccfd-4c42-b7bd-252db3e9ea4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.956864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd56fc-ccfd-4c42-b7bd-252db3e9ea4f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.959725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744ef2-3a7e-43dd-ae0d-20f04add3a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.960464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744ef2-3a7e-43dd-ae0d-20f04add3a38 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.960849 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744ef2-3a7e-43dd-ae0d-20f04add3a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.961252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1744ef2-3a7e-43dd-ae0d-20f04add3a38 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.962112 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1744ef2-3a7e-43dd-ae0d-20f04add3a38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:26.962527 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1744ef2-3a7e-43dd-ae0d-20f04add3a38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.964104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1744ef2-3a7e-43dd-ae0d-20f04add3a38 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d2dd56fc-ccfd-4c42-b7bd-252db3e9ea4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:26.973055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:26.974021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:26.974901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:26.974901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:26.974901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:26.974901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:26.974901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 336/673] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:26.976205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49605c8-a811-43b9-8262-7e3c12c9f075 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.976205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49605c8-a811-43b9-8262-7e3c12c9f075 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.976205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49605c8-a811-43b9-8262-7e3c12c9f075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.987029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 338/674] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:26.987029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744ef2-3a7e-43dd-ae0d-20f04add3a38 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.000858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f49605c8-a811-43b9-8262-7e3c12c9f075 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:27.002015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f49605c8-a811-43b9-8262-7e3c12c9f075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:27.075121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 337/675] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:27.075796 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49605c8-a811-43b9-8262-7e3c12c9f075 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.101856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1744ef2-3a7e-43dd-ae0d-20f04add3a38 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.138305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa25eb-a36f-4234-b535-6b5a6e93b5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.138953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa25eb-a36f-4234-b535-6b5a6e93b5ac None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.139754 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa25eb-a36f-4234-b535-6b5a6e93b5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.140029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa25eb-a36f-4234-b535-6b5a6e93b5ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.150582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa25eb-a36f-4234-b535-6b5a6e93b5ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-22aa25eb-a36f-4234-b535-6b5a6e93b5ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:27.156488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:27.157664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 339/676] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:27.163492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa25eb-a36f-4234-b535-6b5a6e93b5ac None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.187508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:27.191634 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49605c8-a811-43b9-8262-7e3c12c9f075 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.216775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:27.227955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c071f7-40e5-4ef4-8c31-bff92895e674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.228800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c071f7-40e5-4ef4-8c31-bff92895e674 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.229119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c071f7-40e5-4ef4-8c31-bff92895e674 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.229396 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c071f7-40e5-4ef4-8c31-bff92895e674 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.230095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c071f7-40e5-4ef4-8c31-bff92895e674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:27.230404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c071f7-40e5-4ef4-8c31-bff92895e674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.232065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c071f7-40e5-4ef4-8c31-bff92895e674 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.247612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 338/677] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:27.247612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c071f7-40e5-4ef4-8c31-bff92895e674 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.275076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa25eb-a36f-4234-b535-6b5a6e93b5ac None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.312746 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca4165-3ad2-4369-ba0b-2d204d4ec46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.313411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca4165-3ad2-4369-ba0b-2d204d4ec46a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.313692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca4165-3ad2-4369-ba0b-2d204d4ec46a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.313965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca4165-3ad2-4369-ba0b-2d204d4ec46a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.314754 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca4165-3ad2-4369-ba0b-2d204d4ec46a None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=9dbaf1e343474322b2567e2dfda99d4e) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:27.315061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca4165-3ad2-4369-ba0b-2d204d4ec46a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.316491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca4165-3ad2-4369-ba0b-2d204d4ec46a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.332999 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 340/678] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/role_assignments?user.id=9dbaf1e343474322b2567e2dfda99d4e&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:27.332999 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca4165-3ad2-4369-ba0b-2d204d4ec46a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.359651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c071f7-40e5-4ef4-8c31-bff92895e674 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.407643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088faf5e-75f3-4a3c-9264-6ff1bf71de93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.409253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088faf5e-75f3-4a3c-9264-6ff1bf71de93 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.409803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088faf5e-75f3-4a3c-9264-6ff1bf71de93 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.410284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088faf5e-75f3-4a3c-9264-6ff1bf71de93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.434875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088faf5e-75f3-4a3c-9264-6ff1bf71de93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-088faf5e-75f3-4a3c-9264-6ff1bf71de93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:27.443101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:27.444316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:27.445257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 339/679] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:27.445972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088faf5e-75f3-4a3c-9264-6ff1bf71de93 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.469935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca4165-3ad2-4369-ba0b-2d204d4ec46a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.479305 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:27.491381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:27.516195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dc45c-e23a-4511-ae08-f2d0195d5ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.516823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dc45c-e23a-4511-ae08-f2d0195d5ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.517135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dc45c-e23a-4511-ae08-f2d0195d5ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.517415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dc45c-e23a-4511-ae08-f2d0195d5ba7 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.518135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dc45c-e23a-4511-ae08-f2d0195d5ba7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:27.518440 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dc45c-e23a-4511-ae08-f2d0195d5ba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.519797 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dc45c-e23a-4511-ae08-f2d0195d5ba7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.539621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 341/680] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:27.539621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dc45c-e23a-4511-ae08-f2d0195d5ba7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.596607 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088faf5e-75f3-4a3c-9264-6ff1bf71de93 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.635791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2f039-2d50-42e6-b994-ffc5bf8fefd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.636048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2f039-2d50-42e6-b994-ffc5bf8fefd6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.636333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2f039-2d50-42e6-b994-ffc5bf8fefd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.636605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2f039-2d50-42e6-b994-ffc5bf8fefd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.643561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2f039-2d50-42e6-b994-ffc5bf8fefd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.645014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2f039-2d50-42e6-b994-ffc5bf8fefd6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-86c2f039-2d50-42e6-b994-ffc5bf8fefd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:27.658618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:27.659737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:27.660821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:27.660821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:27.660821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:27.660821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:27.660821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:27.660821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:27.660821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:27.660821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:27.660821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 340/681] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:27.660821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2f039-2d50-42e6-b994-ffc5bf8fefd6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.675889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dc45c-e23a-4511-ae08-f2d0195d5ba7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.721328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1874399-e2d3-49fd-9460-c9b28857bab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.722596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1874399-e2d3-49fd-9460-c9b28857bab3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.723470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1874399-e2d3-49fd-9460-c9b28857bab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.723901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1874399-e2d3-49fd-9460-c9b28857bab3 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.725252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1874399-e2d3-49fd-9460-c9b28857bab3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:27.725732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1874399-e2d3-49fd-9460-c9b28857bab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.727650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1874399-e2d3-49fd-9460-c9b28857bab3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.739015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 342/682] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:27.739793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1874399-e2d3-49fd-9460-c9b28857bab3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.755402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:27.788517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2f039-2d50-42e6-b994-ffc5bf8fefd6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.793253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:27.833798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1728fc6-ebf5-4556-ba81-358f1ddb8ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.835259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1728fc6-ebf5-4556-ba81-358f1ddb8ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.835709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1728fc6-ebf5-4556-ba81-358f1ddb8ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.836551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1728fc6-ebf5-4556-ba81-358f1ddb8ca6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.854027 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1728fc6-ebf5-4556-ba81-358f1ddb8ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.856832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1728fc6-ebf5-4556-ba81-358f1ddb8ca6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.862436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1874399-e2d3-49fd-9460-c9b28857bab3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f1728fc6-ebf5-4556-ba81-358f1ddb8ca6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:27.880171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:27.881338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:27.882439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:27.882439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:27.882439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:27.882439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:27.889551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 341/683] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:27.891021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1728fc6-ebf5-4556-ba81-358f1ddb8ca6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.905710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe29a2e-5454-4270-a41f-90fc472340cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.906794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe29a2e-5454-4270-a41f-90fc472340cb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.907178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe29a2e-5454-4270-a41f-90fc472340cb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.907861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe29a2e-5454-4270-a41f-90fc472340cb None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.908449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe29a2e-5454-4270-a41f-90fc472340cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:27.908876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe29a2e-5454-4270-a41f-90fc472340cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.910513 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe29a2e-5454-4270-a41f-90fc472340cb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.924539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 343/684] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:27.925569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe29a2e-5454-4270-a41f-90fc472340cb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:28.029152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:28.030370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1728fc6-ebf5-4556-ba81-358f1ddb8ca6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:28.035352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe29a2e-5454-4270-a41f-90fc472340cb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:28.068418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:28.078335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c05df-8315-419f-81f0-6488a3bca67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:28.079191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c05df-8315-419f-81f0-6488a3bca67c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:28.079476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c05df-8315-419f-81f0-6488a3bca67c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:28.079753 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c05df-8315-419f-81f0-6488a3bca67c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:28.087447 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a2ea0-a927-4586-96c4-e32c36523068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:28.088075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a2ea0-a927-4586-96c4-e32c36523068 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:28.088359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a2ea0-a927-4586-96c4-e32c36523068 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:28.088637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a2ea0-a927-4586-96c4-e32c36523068 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:28.089381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a2ea0-a927-4586-96c4-e32c36523068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:28.097946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a2ea0-a927-4586-96c4-e32c36523068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:28.097946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a2ea0-a927-4586-96c4-e32c36523068 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:28.098177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c05df-8315-419f-81f0-6488a3bca67c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:28.098177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c05df-8315-419f-81f0-6488a3bca67c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:28.108876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 344/685] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:28.108876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a2ea0-a927-4586-96c4-e32c36523068 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e98c05df-8315-419f-81f0-6488a3bca67c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:28.126138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:28.129932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:28.131046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:28.131046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:28.131046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:28.131046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:28.131046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 342/686] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:28.132437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c05df-8315-419f-81f0-6488a3bca67c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:28.239425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a2ea0-a927-4586-96c4-e32c36523068 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:28.262451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c05df-8315-419f-81f0-6488a3bca67c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:28.275745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f0921-a903-4781-9d81-ae1bf5eb888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:28.276372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f0921-a903-4781-9d81-ae1bf5eb888b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:28.276661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f0921-a903-4781-9d81-ae1bf5eb888b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:28.277259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f0921-a903-4781-9d81-ae1bf5eb888b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:28.277785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f0921-a903-4781-9d81-ae1bf5eb888b None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=b038336582a0435dbd47ad2c09cd23e0) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:28.278162 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f0921-a903-4781-9d81-ae1bf5eb888b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:28.279607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f0921-a903-4781-9d81-ae1bf5eb888b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:28.286516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 345/687] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:28 2022] GET /identity/v3/role_assignments?user.id=b038336582a0435dbd47ad2c09cd23e0&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:28.287238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f0921-a903-4781-9d81-ae1bf5eb888b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:28.308562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718a5e2-f8fd-49bf-996b-05f528146ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:28.309298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718a5e2-f8fd-49bf-996b-05f528146ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:28.309643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718a5e2-f8fd-49bf-996b-05f528146ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:28.309981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718a5e2-f8fd-49bf-996b-05f528146ec3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:28.321123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:28.334169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718a5e2-f8fd-49bf-996b-05f528146ec3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:28.335675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718a5e2-f8fd-49bf-996b-05f528146ec3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d718a5e2-f8fd-49bf-996b-05f528146ec3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:28.357018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:28.358085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:28.359402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:28.359402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:28.359402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:28.359402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:28.359402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:28.359402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:28.359402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:28.359402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:28.359402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:28.359402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:28.365545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 343/688] 10.222.0.86 () {56 vars in 1093 bytes} [Thu Jul 28 17:53:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:28.372660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718a5e2-f8fd-49bf-996b-05f528146ec3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:28.434946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f0921-a903-4781-9d81-ae1bf5eb888b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:28.469260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:28.482500 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da493b-de00-452b-a5de-91601644f95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:28.483638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da493b-de00-452b-a5de-91601644f95a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:28.484384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da493b-de00-452b-a5de-91601644f95a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:28.485276 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da493b-de00-452b-a5de-91601644f95a None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:28.486343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da493b-de00-452b-a5de-91601644f95a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:28.487189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da493b-de00-452b-a5de-91601644f95a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:28.489228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da493b-de00-452b-a5de-91601644f95a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:28.507360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 346/689] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:28.515418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da493b-de00-452b-a5de-91601644f95a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:28.555520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718a5e2-f8fd-49bf-996b-05f528146ec3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:28.598562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509e1a1-f3cb-4b80-a930-74e65c9939b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:28.599196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509e1a1-f3cb-4b80-a930-74e65c9939b5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:28.599482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509e1a1-f3cb-4b80-a930-74e65c9939b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:28.599811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509e1a1-f3cb-4b80-a930-74e65c9939b5 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:28.600627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509e1a1-f3cb-4b80-a930-74e65c9939b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:28.600938 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509e1a1-f3cb-4b80-a930-74e65c9939b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:28.602403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509e1a1-f3cb-4b80-a930-74e65c9939b5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:28.624133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 344/690] 10.222.0.86 () {56 vars in 1133 bytes} [Thu Jul 28 17:53:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:28.629967 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509e1a1-f3cb-4b80-a930-74e65c9939b5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:28.635351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:28.656130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da493b-de00-452b-a5de-91601644f95a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:28.699595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75eec7-fa9f-4f20-bd25-328a035717df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:28.700398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75eec7-fa9f-4f20-bd25-328a035717df None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:28.700687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75eec7-fa9f-4f20-bd25-328a035717df None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:28.701024 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75eec7-fa9f-4f20-bd25-328a035717df None admin] PATH_INFO: `/v3/users/swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:28.710571 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75eec7-fa9f-4f20-bd25-328a035717df None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:28.712113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75eec7-fa9f-4f20-bd25-328a035717df None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3c75eec7-fa9f-4f20-bd25-328a035717df None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:28.727019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:28.728028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:28.729089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:28.729089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:28.729089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:28.729089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:28.729089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:28.729089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:28.729089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:28.729089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:28.729089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:28.729089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:28.729638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 347/691] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:28.735673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75eec7-fa9f-4f20-bd25-328a035717df None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:28.759119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:28.768196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509e1a1-f3cb-4b80-a930-74e65c9939b5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:28.842945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdb18d-6199-415c-905b-7422367e8cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:28.844028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdb18d-6199-415c-905b-7422367e8cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:28.845206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdb18d-6199-415c-905b-7422367e8cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:28.846286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdb18d-6199-415c-905b-7422367e8cb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:28.949301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfdb18d-6199-415c-905b-7422367e8cb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:28.951010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfdb18d-6199-415c-905b-7422367e8cb6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:29.052399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75eec7-fa9f-4f20-bd25-328a035717df None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:29.064812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:29.071136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 345/692] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:29.073162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc7e4e-44bb-4851-bd27-0135c82f0d95 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.073633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc7e4e-44bb-4851-bd27-0135c82f0d95 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.074061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc7e4e-44bb-4851-bd27-0135c82f0d95 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.075788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 346/693] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:29.079593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc7e4e-44bb-4851-bd27-0135c82f0d95 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:29.099049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d72de-c9f8-4f0f-8fe9-e07270769618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:29.099993 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d72de-c9f8-4f0f-8fe9-e07270769618 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.100321 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d72de-c9f8-4f0f-8fe9-e07270769618 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.100694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d72de-c9f8-4f0f-8fe9-e07270769618 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.102001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d72de-c9f8-4f0f-8fe9-e07270769618 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:29.102422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d72de-c9f8-4f0f-8fe9-e07270769618 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:29.104015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d72de-c9f8-4f0f-8fe9-e07270769618 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:29.123944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 348/694] 10.222.0.86 () {56 vars in 1135 bytes} [Thu Jul 28 17:53:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:29.125726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bcc5f-a5d3-4e19-bf8f-dcdce001e838 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.126117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bcc5f-a5d3-4e19-bf8f-dcdce001e838 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.126492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bcc5f-a5d3-4e19-bf8f-dcdce001e838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.132891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:29.145643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d95bcc5f-a5d3-4e19-bf8f-dcdce001e838 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:29.146987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d95bcc5f-a5d3-4e19-bf8f-dcdce001e838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:29.191419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc7e4e-44bb-4851-bd27-0135c82f0d95 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:29.221983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 349/695] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:29.223011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bcc5f-a5d3-4e19-bf8f-dcdce001e838 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:29.228351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a2bf5-4299-401d-a538-cc016761aa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:29.229261 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a2bf5-4299-401d-a538-cc016761aa4a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.229698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a2bf5-4299-401d-a538-cc016761aa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.230124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a2bf5-4299-401d-a538-cc016761aa4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.240992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a2bf5-4299-401d-a538-cc016761aa4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:29.242648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a2bf5-4299-401d-a538-cc016761aa4a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-6f0a2bf5-4299-401d-a538-cc016761aa4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:29.259063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:29.260195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:29.261351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:29.261351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:29.261351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:29.261351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:29.263113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 347/696] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:29.265172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a2bf5-4299-401d-a538-cc016761aa4a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:29.325436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:29.337631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bcc5f-a5d3-4e19-bf8f-dcdce001e838 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:29.376092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a2bf5-4299-401d-a538-cc016761aa4a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:29.376636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9ce17-b1a7-4722-a58a-29fa53877c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:29.377625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9ce17-b1a7-4722-a58a-29fa53877c61 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.378126 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9ce17-b1a7-4722-a58a-29fa53877c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.378648 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9ce17-b1a7-4722-a58a-29fa53877c61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.411869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e9077-938d-4a66-9497-ea56a25c45a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:29.412696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e9077-938d-4a66-9497-ea56a25c45a5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.412978 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e9077-938d-4a66-9497-ea56a25c45a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.417522 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:29.424130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e9077-938d-4a66-9497-ea56a25c45a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.436620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e9077-938d-4a66-9497-ea56a25c45a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:29.438148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e9077-938d-4a66-9497-ea56a25c45a5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-005e9077-938d-4a66-9497-ea56a25c45a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:29.453249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:29.454253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:29.455224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:29.455224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:29.455224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:29.455224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:29.455224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 348/697] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:29.455502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e9077-938d-4a66-9497-ea56a25c45a5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:29.466462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9ce17-b1a7-4722-a58a-29fa53877c61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:29.468081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9ce17-b1a7-4722-a58a-29fa53877c61 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:29.563604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e9077-938d-4a66-9497-ea56a25c45a5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:29.566815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:29.582552 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 350/698] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:29.583681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9ce17-b1a7-4722-a58a-29fa53877c61 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:29.603688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502164e-86a8-427d-b599-df834a9ad6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:29.604687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502164e-86a8-427d-b599-df834a9ad6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.605152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502164e-86a8-427d-b599-df834a9ad6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.605579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502164e-86a8-427d-b599-df834a9ad6f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.607369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502164e-86a8-427d-b599-df834a9ad6f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:29.608384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502164e-86a8-427d-b599-df834a9ad6f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:29.609932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502164e-86a8-427d-b599-df834a9ad6f6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:29.617346 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:29.623172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 349/699] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:29.625081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502164e-86a8-427d-b599-df834a9ad6f6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:29.693019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9ce17-b1a7-4722-a58a-29fa53877c61 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:29.726642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f739d92-1cb2-4fb2-9f38-7c271347264a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:29.727285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f739d92-1cb2-4fb2-9f38-7c271347264a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.727591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f739d92-1cb2-4fb2-9f38-7c271347264a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.727842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f739d92-1cb2-4fb2-9f38-7c271347264a None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.728578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f739d92-1cb2-4fb2-9f38-7c271347264a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:29.728876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f739d92-1cb2-4fb2-9f38-7c271347264a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:29.730327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f739d92-1cb2-4fb2-9f38-7c271347264a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:29.733729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502164e-86a8-427d-b599-df834a9ad6f6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:29.746914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 351/700] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:29.746914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299cd16-b8b2-46c1-a004-affe169d8fe8 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.746914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299cd16-b8b2-46c1-a004-affe169d8fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.746914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299cd16-b8b2-46c1-a004-affe169d8fe8 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.746914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 352/701] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:29.746914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299cd16-b8b2-46c1-a004-affe169d8fe8 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:29.776465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e1abd-b6f9-48c4-9aa8-c945fd4291af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:29.777156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e1abd-b6f9-48c4-9aa8-c945fd4291af None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.777443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e1abd-b6f9-48c4-9aa8-c945fd4291af None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.777719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e1abd-b6f9-48c4-9aa8-c945fd4291af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.787954 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e1abd-b6f9-48c4-9aa8-c945fd4291af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:29.789441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e1abd-b6f9-48c4-9aa8-c945fd4291af None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e03e1abd-b6f9-48c4-9aa8-c945fd4291af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:29.804671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:29.805831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:29.812631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:29.812631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:29.812631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:29.812631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:29.812631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 350/702] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:29.812631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e1abd-b6f9-48c4-9aa8-c945fd4291af None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:29.829367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:29.848763 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:29.862274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299cd16-b8b2-46c1-a004-affe169d8fe8 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:29.898442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266eea1-777b-45c4-8ac1-338058ee5174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:29.899188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266eea1-777b-45c4-8ac1-338058ee5174 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.899540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266eea1-777b-45c4-8ac1-338058ee5174 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.899904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266eea1-777b-45c4-8ac1-338058ee5174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.906890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266eea1-777b-45c4-8ac1-338058ee5174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:29.908342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266eea1-777b-45c4-8ac1-338058ee5174 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:29.916337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e1abd-b6f9-48c4-9aa8-c945fd4291af None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-f266eea1-777b-45c4-8ac1-338058ee5174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:29.919482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:29.920404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:29.921332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:29.921332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:29.921332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:29.921332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:29.921332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:29.921332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:29.921332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:29.921332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:29.921332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 353/703] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:29.922107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b15dc-9149-4e03-841d-50857acbd724 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.922197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b15dc-9149-4e03-841d-50857acbd724 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.922459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b15dc-9149-4e03-841d-50857acbd724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.941077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d28b15dc-9149-4e03-841d-50857acbd724 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:29.942265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d28b15dc-9149-4e03-841d-50857acbd724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:29.958496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dc4b5-647d-4c89-9d27-f145536bd1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:29.959118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dc4b5-647d-4c89-9d27-f145536bd1da None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:29.959505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dc4b5-647d-4c89-9d27-f145536bd1da None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:29.959677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dc4b5-647d-4c89-9d27-f145536bd1da None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:29.960352 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40dc4b5-647d-4c89-9d27-f145536bd1da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:29.961810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40dc4b5-647d-4c89-9d27-f145536bd1da None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:30.010485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 354/704] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:30.011138 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b15dc-9149-4e03-841d-50857acbd724 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:30.089082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 351/705] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:53:29 2022] POST /identity/v3/users => generated 254 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:30.089082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dc4b5-647d-4c89-9d27-f145536bd1da None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:30.115250 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:30.141383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b15dc-9149-4e03-841d-50857acbd724 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:30.195460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:30.201789 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae180d1-b15c-42ce-98bf-7aae7d8bf15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:30.202447 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae180d1-b15c-42ce-98bf-7aae7d8bf15f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:30.202732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae180d1-b15c-42ce-98bf-7aae7d8bf15f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:30.203011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae180d1-b15c-42ce-98bf-7aae7d8bf15f None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:30.203737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae180d1-b15c-42ce-98bf-7aae7d8bf15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:30.204470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae180d1-b15c-42ce-98bf-7aae7d8bf15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:30.206290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae180d1-b15c-42ce-98bf-7aae7d8bf15f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:30.225137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 355/706] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:30.226062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae180d1-b15c-42ce-98bf-7aae7d8bf15f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:30.239012 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dc4b5-647d-4c89-9d27-f145536bd1da None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:30.287482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c541e-1eea-4a40-8134-68333a6248f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:30.288891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c541e-1eea-4a40-8134-68333a6248f1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:30.289487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c541e-1eea-4a40-8134-68333a6248f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:30.290062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c541e-1eea-4a40-8134-68333a6248f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:30.290901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c541e-1eea-4a40-8134-68333a6248f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:30.291284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c541e-1eea-4a40-8134-68333a6248f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:30.292781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c541e-1eea-4a40-8134-68333a6248f1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:30.310295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 352/707] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:30.311816 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c541e-1eea-4a40-8134-68333a6248f1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:30.356925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae180d1-b15c-42ce-98bf-7aae7d8bf15f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:30.394998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cb577-a8e0-41e3-91e0-77d72203f65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:30.395886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cb577-a8e0-41e3-91e0-77d72203f65f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:30.396327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cb577-a8e0-41e3-91e0-77d72203f65f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:30.396747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cb577-a8e0-41e3-91e0-77d72203f65f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:30.397547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cb577-a8e0-41e3-91e0-77d72203f65f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:30.399222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cb577-a8e0-41e3-91e0-77d72203f65f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-1a9cb577-a8e0-41e3-91e0-77d72203f65f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:30.411622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:30.412805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:53:30.413905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:30.416573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 356/708] 10.222.0.86 () {56 vars in 1129 bytes} [Thu Jul 28 17:53:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:30.421956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cb577-a8e0-41e3-91e0-77d72203f65f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:30.437052 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:30.454944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c541e-1eea-4a40-8134-68333a6248f1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:30.490941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:30.496642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ee0b0-3381-4ec8-bacc-b331642a8dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:30.497320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ee0b0-3381-4ec8-bacc-b331642a8dde None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:30.497601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ee0b0-3381-4ec8-bacc-b331642a8dde None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:30.497920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ee0b0-3381-4ec8-bacc-b331642a8dde None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:30.505848 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ee0b0-3381-4ec8-bacc-b331642a8dde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:30.509457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ee0b0-3381-4ec8-bacc-b331642a8dde None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-770ee0b0-3381-4ec8-bacc-b331642a8dde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:30.520481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:30.521521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:30.522724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:30.522724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:30.522724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:30.522724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:30.522724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:30.522724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:30.522724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:30.522724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:30.526413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 353/709] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:30.527457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ee0b0-3381-4ec8-bacc-b331642a8dde None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:30.564524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cb577-a8e0-41e3-91e0-77d72203f65f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:30.602624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728b943-e52f-46b4-b2a1-cbe4b2737947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:30.603596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728b943-e52f-46b4-b2a1-cbe4b2737947 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:30.604061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728b943-e52f-46b4-b2a1-cbe4b2737947 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:30.604557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728b943-e52f-46b4-b2a1-cbe4b2737947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:30.605674 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728b943-e52f-46b4-b2a1-cbe4b2737947 None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=7d7bcb4e7c434622b650cdfc500a9680) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:30.606197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728b943-e52f-46b4-b2a1-cbe4b2737947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:30.608470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728b943-e52f-46b4-b2a1-cbe4b2737947 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:30.615565 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 357/710] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:30 2022] GET /identity/v3/role_assignments?user.id=7d7bcb4e7c434622b650cdfc500a9680&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:30.616691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728b943-e52f-46b4-b2a1-cbe4b2737947 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:30.668728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ee0b0-3381-4ec8-bacc-b331642a8dde None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:30.725068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:30.742606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a690ac2-0807-4d38-933b-9b88c24c3d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:30.743833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a690ac2-0807-4d38-933b-9b88c24c3d0d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:30.744640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a690ac2-0807-4d38-933b-9b88c24c3d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:30.745184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a690ac2-0807-4d38-933b-9b88c24c3d0d None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:30.746100 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a690ac2-0807-4d38-933b-9b88c24c3d0d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:30.746638 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a690ac2-0807-4d38-933b-9b88c24c3d0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:30.749660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a690ac2-0807-4d38-933b-9b88c24c3d0d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:30.757241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 354/711] 10.222.0.86 () {56 vars in 1124 bytes} [Thu Jul 28 17:53:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:30.759596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db5b1b-29ee-40dc-9f25-49e42052c9bd None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:30.760110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db5b1b-29ee-40dc-9f25-49e42052c9bd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:30.760627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db5b1b-29ee-40dc-9f25-49e42052c9bd None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:30.761220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728b943-e52f-46b4-b2a1-cbe4b2737947 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:30.763076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 355/712] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:30.765118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db5b1b-29ee-40dc-9f25-49e42052c9bd None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:30.818846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c258e3-f673-4015-aa85-3daa8e782c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:30.819682 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c258e3-f673-4015-aa85-3daa8e782c59 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:30.820021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c258e3-f673-4015-aa85-3daa8e782c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:30.820359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c258e3-f673-4015-aa85-3daa8e782c59 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:30.821261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c258e3-f673-4015-aa85-3daa8e782c59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:30.821616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c258e3-f673-4015-aa85-3daa8e782c59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:30.827191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c258e3-f673-4015-aa85-3daa8e782c59 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:30.841026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 358/713] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:30.841026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7092708-f87e-4989-b337-35037b93bc4a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:30.841026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7092708-f87e-4989-b337-35037b93bc4a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:30.841026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7092708-f87e-4989-b337-35037b93bc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:30.855901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:30.867947 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f7092708-f87e-4989-b337-35037b93bc4a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:30.867947 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f7092708-f87e-4989-b337-35037b93bc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:30.910636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db5b1b-29ee-40dc-9f25-49e42052c9bd None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:30.947786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8139eb-53e1-4379-a23a-8aab01d0dfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:30.948573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8139eb-53e1-4379-a23a-8aab01d0dfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:30.948959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8139eb-53e1-4379-a23a-8aab01d0dfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:30.949358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8139eb-53e1-4379-a23a-8aab01d0dfc9 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:30.950202 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8139eb-53e1-4379-a23a-8aab01d0dfc9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:30.950647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8139eb-53e1-4379-a23a-8aab01d0dfc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:30.952172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8139eb-53e1-4379-a23a-8aab01d0dfc9 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:30.957039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 359/714] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:30.958258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7092708-f87e-4989-b337-35037b93bc4a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:30.973034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 356/715] 10.222.0.86 () {56 vars in 1124 bytes} [Thu Jul 28 17:53:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:30.981811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8139eb-53e1-4379-a23a-8aab01d0dfc9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:31.055478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:31.128094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7092708-f87e-4989-b337-35037b93bc4a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:31.176043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b59bc-c3d4-449f-86d6-b52840b4b545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:31.176683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b59bc-c3d4-449f-86d6-b52840b4b545 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:31.176991 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b59bc-c3d4-449f-86d6-b52840b4b545 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:31.177279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b59bc-c3d4-449f-86d6-b52840b4b545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:31.178091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b59bc-c3d4-449f-86d6-b52840b4b545 None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=3077d844257c48b3b4ac2db9177fb6df) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:31.178404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b59bc-c3d4-449f-86d6-b52840b4b545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:31.179875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b59bc-c3d4-449f-86d6-b52840b4b545 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:31.187883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 360/716] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:30 2022] GET /identity/v3/role_assignments?user.id=3077d844257c48b3b4ac2db9177fb6df&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:31.210522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8139eb-53e1-4379-a23a-8aab01d0dfc9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:31.234659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:31.250166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1754f81-0d6e-4d05-84fd-0314b96e30cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:31.251100 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1754f81-0d6e-4d05-84fd-0314b96e30cd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:31.251389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1754f81-0d6e-4d05-84fd-0314b96e30cd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:31.251667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1754f81-0d6e-4d05-84fd-0314b96e30cd None admin] PATH_INFO: `/v3/services/network` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:31.252397 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1754f81-0d6e-4d05-84fd-0314b96e30cd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:31.253906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1754f81-0d6e-4d05-84fd-0314b96e30cd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d1754f81-0d6e-4d05-84fd-0314b96e30cd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:31.266536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:31.267627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:31.268463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 357/717] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:30 2022] GET /identity/v3/services/network => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:31.274091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b59bc-c3d4-449f-86d6-b52840b4b545 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:31.387557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b59bc-c3d4-449f-86d6-b52840b4b545 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:31.425414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:31.436502 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1a68e-bb7a-46d8-a036-9abf94403e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:31.437178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1a68e-bb7a-46d8-a036-9abf94403e77 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:31.437471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1a68e-bb7a-46d8-a036-9abf94403e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:31.437826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1a68e-bb7a-46d8-a036-9abf94403e77 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:31.438524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1a68e-bb7a-46d8-a036-9abf94403e77 None admin] RBAC: Adding query filter params (name=network) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:31.438840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1a68e-bb7a-46d8-a036-9abf94403e77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:31.440232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1a68e-bb7a-46d8-a036-9abf94403e77 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:31.445594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 361/718] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:53:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:31.451514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1754f81-0d6e-4d05-84fd-0314b96e30cd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:31.587003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1754f81-0d6e-4d05-84fd-0314b96e30cd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:31.629873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dbc8f-7a4d-419b-94bb-ff5e75f4bac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:31.630500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dbc8f-7a4d-419b-94bb-ff5e75f4bac0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:31.630785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dbc8f-7a4d-419b-94bb-ff5e75f4bac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:31.631066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dbc8f-7a4d-419b-94bb-ff5e75f4bac0 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:31.631753 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dbc8f-7a4d-419b-94bb-ff5e75f4bac0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:31.632064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dbc8f-7a4d-419b-94bb-ff5e75f4bac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:31.633470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dbc8f-7a4d-419b-94bb-ff5e75f4bac0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:31.638615 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 358/719] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:53:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:32.311215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e54336-875c-43a1-8976-f9658bffa12c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:32.311832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e54336-875c-43a1-8976-f9658bffa12c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:32.311832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e54336-875c-43a1-8976-f9658bffa12c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:32.313418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 362/720] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:32.318975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eef69-ece9-4e2f-8190-0d140ee5a289 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:32.319269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eef69-ece9-4e2f-8190-0d140ee5a289 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:32.319550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eef69-ece9-4e2f-8190-0d140ee5a289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:32.346051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-357eef69-ece9-4e2f-8190-0d140ee5a289 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:32.347281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-357eef69-ece9-4e2f-8190-0d140ee5a289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:32.408680 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:32.452804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 359/721] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:53:32.491311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e54336-875c-43a1-8976-f9658bffa12c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:32.631563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e54336-875c-43a1-8976-f9658bffa12c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:32.673593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102403e-7070-4884-a3d5-a307336bc30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:32.674546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102403e-7070-4884-a3d5-a307336bc30f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:32.674983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102403e-7070-4884-a3d5-a307336bc30f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:32.675419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102403e-7070-4884-a3d5-a307336bc30f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:32.686596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102403e-7070-4884-a3d5-a307336bc30f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3102403e-7070-4884-a3d5-a307336bc30f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:32.692614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:32.693773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:32.694801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:32.697051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 363/722] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:32.704996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eef69-ece9-4e2f-8190-0d140ee5a289 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:32.970043 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eef69-ece9-4e2f-8190-0d140ee5a289 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:33.019065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a0959-ffce-4df8-8774-f34ab48efee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:33.019701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0959-ffce-4df8-8774-f34ab48efee0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:33.019989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0959-ffce-4df8-8774-f34ab48efee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:33.020258 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0959-ffce-4df8-8774-f34ab48efee0 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.020999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a0959-ffce-4df8-8774-f34ab48efee0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:33.021314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a0959-ffce-4df8-8774-f34ab48efee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:33.022683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a0959-ffce-4df8-8774-f34ab48efee0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:33.037254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 360/723] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:33.042679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102403e-7070-4884-a3d5-a307336bc30f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:33.080687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d27501-8f0b-4c9a-a496-3d737bcab500 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:33.081040 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d27501-8f0b-4c9a-a496-3d737bcab500 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:33.081271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d27501-8f0b-4c9a-a496-3d737bcab500 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.083087 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 361/724] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:33.089006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195b6af-5f50-421a-ae59-1fbda6d87719 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:33.089006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195b6af-5f50-421a-ae59-1fbda6d87719 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:33.089006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195b6af-5f50-421a-ae59-1fbda6d87719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.149367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:33.150179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9195b6af-5f50-421a-ae59-1fbda6d87719 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:33.151424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9195b6af-5f50-421a-ae59-1fbda6d87719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:33.212350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102403e-7070-4884-a3d5-a307336bc30f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:33.239280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:33.260296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07966e6d-1ffe-4cf4-98af-bdd0dac2d0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:33.261354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07966e6d-1ffe-4cf4-98af-bdd0dac2d0ec None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:33.261917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07966e6d-1ffe-4cf4-98af-bdd0dac2d0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:33.262203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07966e6d-1ffe-4cf4-98af-bdd0dac2d0ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.282772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 362/725] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:53:33.286165 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07966e6d-1ffe-4cf4-98af-bdd0dac2d0ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:33.287652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07966e6d-1ffe-4cf4-98af-bdd0dac2d0ec None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-07966e6d-1ffe-4cf4-98af-bdd0dac2d0ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:33.302924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:33.303986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:33.305617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:33.305617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:33.305617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:33.305617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:33.305617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 364/726] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:33.310090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195b6af-5f50-421a-ae59-1fbda6d87719 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:33.324236 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07966e6d-1ffe-4cf4-98af-bdd0dac2d0ec None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:33.469255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:33.474795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07966e6d-1ffe-4cf4-98af-bdd0dac2d0ec None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:33.511180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195b6af-5f50-421a-ae59-1fbda6d87719 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:33.556392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5d8ec-1734-4777-a66d-25d451810359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:33.557425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d8ec-1734-4777-a66d-25d451810359 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:33.557877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d8ec-1734-4777-a66d-25d451810359 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:33.558330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d8ec-1734-4777-a66d-25d451810359 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.559441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5d8ec-1734-4777-a66d-25d451810359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:33.559949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5d8ec-1734-4777-a66d-25d451810359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:33.562114 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5d8ec-1734-4777-a66d-25d451810359 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:33.584495 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713834e0-94bf-4a03-b5b8-7b6e6b7f5af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:33.585179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713834e0-94bf-4a03-b5b8-7b6e6b7f5af8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:33.585466 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713834e0-94bf-4a03-b5b8-7b6e6b7f5af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:33.585803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713834e0-94bf-4a03-b5b8-7b6e6b7f5af8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.587240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 363/727] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:33.593193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5d8ec-1734-4777-a66d-25d451810359 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:33.611547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713834e0-94bf-4a03-b5b8-7b6e6b7f5af8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-713834e0-94bf-4a03-b5b8-7b6e6b7f5af8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:33.625523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:33.626592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:33.627433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:33.630817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 365/728] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:33.635082 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713834e0-94bf-4a03-b5b8-7b6e6b7f5af8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:33.655824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:33.767654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713834e0-94bf-4a03-b5b8-7b6e6b7f5af8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:33.837019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5d8ec-1734-4777-a66d-25d451810359 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:33.858352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9373125-aa2a-4d1d-89c2-f2fcda842274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:33.859004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9373125-aa2a-4d1d-89c2-f2fcda842274 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:33.859294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9373125-aa2a-4d1d-89c2-f2fcda842274 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:33.859411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63c51d-bb49-4924-8983-f012366f6eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:33.859566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9373125-aa2a-4d1d-89c2-f2fcda842274 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.860192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c51d-bb49-4924-8983-f012366f6eab None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:33.860315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9373125-aa2a-4d1d-89c2-f2fcda842274 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:33.860543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c51d-bb49-4924-8983-f012366f6eab None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:33.860653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9373125-aa2a-4d1d-89c2-f2fcda842274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:33.860937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c51d-bb49-4924-8983-f012366f6eab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.861999 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9373125-aa2a-4d1d-89c2-f2fcda842274 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:33.877895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63c51d-bb49-4924-8983-f012366f6eab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:33.879533 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 366/729] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:33.880431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63c51d-bb49-4924-8983-f012366f6eab None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:33.880866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00679-b51e-42c6-a0dd-aeb01f25b9f0 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:33.881927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00679-b51e-42c6-a0dd-aeb01f25b9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:33.882217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00679-b51e-42c6-a0dd-aeb01f25b9f0 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.887442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 367/730] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:33.887442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00679-b51e-42c6-a0dd-aeb01f25b9f0 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-1e63c51d-bb49-4924-8983-f012366f6eab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:33.910910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:33.911957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:33.912898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:33.912898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:33.912898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:33.912898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:33.912898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:33.912898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:33.912898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:33.912898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:33.912898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:53:33.912898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:33.916097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 364/731] 10.222.0.86 () {56 vars in 1117 bytes} [Thu Jul 28 17:53:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:33.917026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:33.919751 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea291f-a230-4a59-80cf-b8b0aa197b43 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:33.920840 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea291f-a230-4a59-80cf-b8b0aa197b43 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:33.922903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea291f-a230-4a59-80cf-b8b0aa197b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:33.952745 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-28ea291f-a230-4a59-80cf-b8b0aa197b43 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:33.954353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-28ea291f-a230-4a59-80cf-b8b0aa197b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:34.080374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 365/732] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:34.080374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea291f-a230-4a59-80cf-b8b0aa197b43 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:34.093805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00679-b51e-42c6-a0dd-aeb01f25b9f0 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:34.104357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:34.157984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69fde7-42f0-443a-9b37-a925899e0d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:34.158642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69fde7-42f0-443a-9b37-a925899e0d35 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.158930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69fde7-42f0-443a-9b37-a925899e0d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.159210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69fde7-42f0-443a-9b37-a925899e0d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.171292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69fde7-42f0-443a-9b37-a925899e0d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:34.172803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69fde7-42f0-443a-9b37-a925899e0d35 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:34.185456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea291f-a230-4a59-80cf-b8b0aa197b43 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-5c69fde7-42f0-443a-9b37-a925899e0d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:34.188920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:34.189868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:34.191441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:34.191441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:34.191441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:34.191441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:34.191441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 368/733] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:34.191730 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0c479-3bf0-4ec8-b15b-dc98f5881496 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.192028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0c479-3bf0-4ec8-b15b-dc98f5881496 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.192312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0c479-3bf0-4ec8-b15b-dc98f5881496 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.197613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 369/734] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:34.198267 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0c479-3bf0-4ec8-b15b-dc98f5881496 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:34.226596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8741b-be50-4122-933d-05410fb8097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:34.227228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8741b-be50-4122-933d-05410fb8097f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.227508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8741b-be50-4122-933d-05410fb8097f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.227787 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8741b-be50-4122-933d-05410fb8097f None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.228587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8741b-be50-4122-933d-05410fb8097f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:34.228898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8741b-be50-4122-933d-05410fb8097f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:34.230360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8741b-be50-4122-933d-05410fb8097f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:34.247977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 366/735] 10.222.0.86 () {56 vars in 1149 bytes} [Thu Jul 28 17:53:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:34.248803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8741b-be50-4122-933d-05410fb8097f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:34.275839 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:34.301851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0c479-3bf0-4ec8-b15b-dc98f5881496 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:34.325406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:34.336431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2dd7f-f5c1-49ec-9939-957bc435cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:34.337011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2dd7f-f5c1-49ec-9939-957bc435cc2e None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.337298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2dd7f-f5c1-49ec-9939-957bc435cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.337575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2dd7f-f5c1-49ec-9939-957bc435cc2e None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.338266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2dd7f-f5c1-49ec-9939-957bc435cc2e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:34.339628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2dd7f-f5c1-49ec-9939-957bc435cc2e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:34.346942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8741b-be50-4122-933d-05410fb8097f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:34.380954 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70460b9c-ad0f-418d-bbc8-aa2949702699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:34.381624 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70460b9c-ad0f-418d-bbc8-aa2949702699 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.381930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70460b9c-ad0f-418d-bbc8-aa2949702699 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.382207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70460b9c-ad0f-418d-bbc8-aa2949702699 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.382900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70460b9c-ad0f-418d-bbc8-aa2949702699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:34.383204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70460b9c-ad0f-418d-bbc8-aa2949702699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:34.384578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70460b9c-ad0f-418d-bbc8-aa2949702699 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:34.395654 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 367/736] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:34.396594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a60626-41d7-48b9-9948-2487aa1f091d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.396881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a60626-41d7-48b9-9948-2487aa1f091d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.397183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a60626-41d7-48b9-9948-2487aa1f091d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.415730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-30a60626-41d7-48b9-9948-2487aa1f091d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:34.416931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-30a60626-41d7-48b9-9948-2487aa1f091d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:34.441809 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 370/737] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:53:34 2022] POST /identity/v3/services => generated 251 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:34.443150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbddb07-10b0-4179-a231-f856ffbb2bce None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.443448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbddb07-10b0-4179-a231-f856ffbb2bce None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.443726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbddb07-10b0-4179-a231-f856ffbb2bce None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.445049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 371/738] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:34.445774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbddb07-10b0-4179-a231-f856ffbb2bce None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:34.513093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 368/739] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:34.514039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a60626-41d7-48b9-9948-2487aa1f091d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:34.584177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbddb07-10b0-4179-a231-f856ffbb2bce None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:34.600825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:34.629390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b617c-5944-4aa9-9a87-6bf2f8cd781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:34.630032 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b617c-5944-4aa9-9a87-6bf2f8cd781c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.630315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b617c-5944-4aa9-9a87-6bf2f8cd781c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.630590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b617c-5944-4aa9-9a87-6bf2f8cd781c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.634239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:34.646734 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a60626-41d7-48b9-9948-2487aa1f091d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:34.694062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc20e5d-ae49-4a1a-88a6-394c5a24ce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:34.694719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc20e5d-ae49-4a1a-88a6-394c5a24ce88 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.695005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc20e5d-ae49-4a1a-88a6-394c5a24ce88 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.695281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc20e5d-ae49-4a1a-88a6-394c5a24ce88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.703680 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc20e5d-ae49-4a1a-88a6-394c5a24ce88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:34.705121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc20e5d-ae49-4a1a-88a6-394c5a24ce88 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:34.707833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b617c-5944-4aa9-9a87-6bf2f8cd781c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:34.709344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b617c-5944-4aa9-9a87-6bf2f8cd781c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-1fc20e5d-ae49-4a1a-88a6-394c5a24ce88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:34.718831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:34.719782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:34.720732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:34.720732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:34.720732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:34.720732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:34.720732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:34.720732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:34.720732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:34.720732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:34.720732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:34.720732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:34.720732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 369/740] 10.222.0.86 () {56 vars in 1093 bytes} [Thu Jul 28 17:53:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:34.721913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed918001-2ff8-4745-964c-8301d88616a5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.721913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed918001-2ff8-4745-964c-8301d88616a5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.722156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed918001-2ff8-4745-964c-8301d88616a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.743841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ed918001-2ff8-4745-964c-8301d88616a5 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:34.745749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ed918001-2ff8-4745-964c-8301d88616a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:34.817834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 370/741] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:53:34.819035 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed918001-2ff8-4745-964c-8301d88616a5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:34.819804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 372/742] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:34 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:53:34.820655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b617c-5944-4aa9-9a87-6bf2f8cd781c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:34.864098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:34.885460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:34.934851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed918001-2ff8-4745-964c-8301d88616a5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:34.938316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b617c-5944-4aa9-9a87-6bf2f8cd781c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:34.978829 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba17b40-bc2f-4fc8-97da-0252024f3be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:34.979453 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba17b40-bc2f-4fc8-97da-0252024f3be6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.979741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba17b40-bc2f-4fc8-97da-0252024f3be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.980003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba17b40-bc2f-4fc8-97da-0252024f3be6 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.980823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba17b40-bc2f-4fc8-97da-0252024f3be6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:34.981141 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba17b40-bc2f-4fc8-97da-0252024f3be6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:34.982634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba17b40-bc2f-4fc8-97da-0252024f3be6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:34.984468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236739b-9154-4263-9742-151ac3737eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:34.985147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236739b-9154-4263-9742-151ac3737eca None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:34.985437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236739b-9154-4263-9742-151ac3737eca None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:34.985712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236739b-9154-4263-9742-151ac3737eca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:34.996744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236739b-9154-4263-9742-151ac3737eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:35.005690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 373/743] 10.222.0.86 () {56 vars in 1133 bytes} [Thu Jul 28 17:53:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d236739b-9154-4263-9742-151ac3737eca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:35.005863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:35.006770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:35.007608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:35.008251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba17b40-bc2f-4fc8-97da-0252024f3be6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:35.008321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 371/744] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:35.008321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236739b-9154-4263-9742-151ac3737eca None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:35.108557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:35.115417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236739b-9154-4263-9742-151ac3737eca None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:35.117953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba17b40-bc2f-4fc8-97da-0252024f3be6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:35.131247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:35.152563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7214c-41d9-46d0-ae25-9a63cbb90900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:35.153212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7214c-41d9-46d0-ae25-9a63cbb90900 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:35.153496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7214c-41d9-46d0-ae25-9a63cbb90900 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:35.153820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7214c-41d9-46d0-ae25-9a63cbb90900 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:35.154980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7214c-41d9-46d0-ae25-9a63cbb90900 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:35.155679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961834f2-3f16-42d8-b511-07bc4cd55793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:35.156341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7214c-41d9-46d0-ae25-9a63cbb90900 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:35.156442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961834f2-3f16-42d8-b511-07bc4cd55793 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:35.156763 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961834f2-3f16-42d8-b511-07bc4cd55793 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:35.157145 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961834f2-3f16-42d8-b511-07bc4cd55793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:35.168109 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961834f2-3f16-42d8-b511-07bc4cd55793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:35.169649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961834f2-3f16-42d8-b511-07bc4cd55793 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-961834f2-3f16-42d8-b511-07bc4cd55793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:35.185465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:35.186573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:35.187505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:35.187505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:35.187505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:35.187505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:35.187505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 374/745] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:35.187799 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961834f2-3f16-42d8-b511-07bc4cd55793 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:35.254040 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 372/746] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:53:35 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:35.254040 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7214c-41d9-46d0-ae25-9a63cbb90900 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:35.327402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961834f2-3f16-42d8-b511-07bc4cd55793 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:35.391308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7214c-41d9-46d0-ae25-9a63cbb90900 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:35.393151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cd84d-c2fc-4526-af8c-e0470a5534f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:35.394242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd84d-c2fc-4526-af8c-e0470a5534f4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:35.394629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd84d-c2fc-4526-af8c-e0470a5534f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:35.395060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd84d-c2fc-4526-af8c-e0470a5534f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:35.407321 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:35.410251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cd84d-c2fc-4526-af8c-e0470a5534f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:35.412717 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cd84d-c2fc-4526-af8c-e0470a5534f4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8f0cd84d-c2fc-4526-af8c-e0470a5534f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:35.438511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:35.439438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:35.440417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:35.440417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:35.440417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:35.440417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:35.440417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 375/747] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:35.440694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cd84d-c2fc-4526-af8c-e0470a5534f4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:35.448799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90a883-ed6e-4096-9021-ae0f08ab4577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:35.449801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90a883-ed6e-4096-9021-ae0f08ab4577 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:35.450081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90a883-ed6e-4096-9021-ae0f08ab4577 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:35.450355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90a883-ed6e-4096-9021-ae0f08ab4577 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:35.451088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90a883-ed6e-4096-9021-ae0f08ab4577 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:35.451462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90a883-ed6e-4096-9021-ae0f08ab4577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:35.452828 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90a883-ed6e-4096-9021-ae0f08ab4577 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:35.463718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 373/748] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:35.464544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90a883-ed6e-4096-9021-ae0f08ab4577 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:35.502314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:35.575348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90a883-ed6e-4096-9021-ae0f08ab4577 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:35.586145 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cd84d-c2fc-4526-af8c-e0470a5534f4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:35.625673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7ab55-2f33-485d-94e8-6ac09d3cc9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:35.626490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7ab55-2f33-485d-94e8-6ac09d3cc9cd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:35.626778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7ab55-2f33-485d-94e8-6ac09d3cc9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:35.627448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7ab55-2f33-485d-94e8-6ac09d3cc9cd None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:35.628513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7ab55-2f33-485d-94e8-6ac09d3cc9cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:35.628827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7ab55-2f33-485d-94e8-6ac09d3cc9cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:35.631104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7ab55-2f33-485d-94e8-6ac09d3cc9cd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:35.636093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b7b6c-d1c0-4b7a-abf2-b45c5ccd39d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:35.636728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7b6c-d1c0-4b7a-abf2-b45c5ccd39d2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:35.637031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7b6c-d1c0-4b7a-abf2-b45c5ccd39d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:35.637309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7b6c-d1c0-4b7a-abf2-b45c5ccd39d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:35.638013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b7b6c-d1c0-4b7a-abf2-b45c5ccd39d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:35.638318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b7b6c-d1c0-4b7a-abf2-b45c5ccd39d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:35.640375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b7b6c-d1c0-4b7a-abf2-b45c5ccd39d2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:35.645191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 374/749] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:35.646085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7ab55-2f33-485d-94e8-6ac09d3cc9cd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:35.655863 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 376/750] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:35.656720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b7b6c-d1c0-4b7a-abf2-b45c5ccd39d2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:35.733578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:35.760313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:35.773391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7ab55-2f33-485d-94e8-6ac09d3cc9cd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:35.794456 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b7b6c-d1c0-4b7a-abf2-b45c5ccd39d2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:35.810083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadb296-5d2d-4316-a833-c88bb936db19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:35.810887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadb296-5d2d-4316-a833-c88bb936db19 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:35.811285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadb296-5d2d-4316-a833-c88bb936db19 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:35.811625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadb296-5d2d-4316-a833-c88bb936db19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:35.822981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadb296-5d2d-4316-a833-c88bb936db19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:35.824247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadb296-5d2d-4316-a833-c88bb936db19 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-8aadb296-5d2d-4316-a833-c88bb936db19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:35.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:35.839930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:35.841069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:35.841069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:35.841069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:35.841069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:35.841780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fef6e-a8eb-4a61-a87f-77d71195b87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:35.842524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 375/751] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:35.843445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadb296-5d2d-4316-a833-c88bb936db19 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:35.843636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fef6e-a8eb-4a61-a87f-77d71195b87a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:35.844078 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fef6e-a8eb-4a61-a87f-77d71195b87a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:35.844515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fef6e-a8eb-4a61-a87f-77d71195b87a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:35.853785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1fef6e-a8eb-4a61-a87f-77d71195b87a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:35.856015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1fef6e-a8eb-4a61-a87f-77d71195b87a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-be1fef6e-a8eb-4a61-a87f-77d71195b87a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:35.871485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:35.872417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:35.873960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:35.873960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:35.873960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:35.873960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:35.873960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:35.873960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:35.873960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:35.873960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:35.875834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 377/752] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:35.877430 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fef6e-a8eb-4a61-a87f-77d71195b87a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:35.963538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadb296-5d2d-4316-a833-c88bb936db19 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:36.007814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be4199-016d-43b9-9907-fda14eafec8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:36.009109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be4199-016d-43b9-9907-fda14eafec8d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.010004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be4199-016d-43b9-9907-fda14eafec8d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.010619 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be4199-016d-43b9-9907-fda14eafec8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.012794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fef6e-a8eb-4a61-a87f-77d71195b87a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:36.019336 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be4199-016d-43b9-9907-fda14eafec8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:36.021535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be4199-016d-43b9-9907-fda14eafec8d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:36.026965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d1be4199-016d-43b9-9907-fda14eafec8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:36.039009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:36.046443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:36.047535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:36.047535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:36.047535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:36.047535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:36.047535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:36.047535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:36.047535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:36.047535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:36.047535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 376/753] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:36.047535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be4199-016d-43b9-9907-fda14eafec8d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:36.052301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:36.055941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110fc83-0283-4b3a-9c29-a3a2ca24d6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:36.056650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110fc83-0283-4b3a-9c29-a3a2ca24d6fe None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.057088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110fc83-0283-4b3a-9c29-a3a2ca24d6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.057437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110fc83-0283-4b3a-9c29-a3a2ca24d6fe None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.058229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110fc83-0283-4b3a-9c29-a3a2ca24d6fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:36.058606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110fc83-0283-4b3a-9c29-a3a2ca24d6fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:36.060156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110fc83-0283-4b3a-9c29-a3a2ca24d6fe None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:36.073778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 378/754] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:36.074723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110fc83-0283-4b3a-9c29-a3a2ca24d6fe None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:36.177890 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be4199-016d-43b9-9907-fda14eafec8d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:36.186913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110fc83-0283-4b3a-9c29-a3a2ca24d6fe None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:36.215674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad472aee-7222-4d44-b80e-501447a007dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:36.216581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad472aee-7222-4d44-b80e-501447a007dc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.217804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad472aee-7222-4d44-b80e-501447a007dc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.218383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad472aee-7222-4d44-b80e-501447a007dc None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.219329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad472aee-7222-4d44-b80e-501447a007dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:36.219835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad472aee-7222-4d44-b80e-501447a007dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:36.221523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad472aee-7222-4d44-b80e-501447a007dc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:36.223318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafecf18-74e4-4f0a-a0a9-ade8e2cb0a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:36.223843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafecf18-74e4-4f0a-a0a9-ade8e2cb0a70 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.224127 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafecf18-74e4-4f0a-a0a9-ade8e2cb0a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.224401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafecf18-74e4-4f0a-a0a9-ade8e2cb0a70 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.225439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafecf18-74e4-4f0a-a0a9-ade8e2cb0a70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:36.225549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafecf18-74e4-4f0a-a0a9-ade8e2cb0a70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:36.227079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafecf18-74e4-4f0a-a0a9-ade8e2cb0a70 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:36.237032 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 377/755] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:36.238182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad472aee-7222-4d44-b80e-501447a007dc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:36.245141 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 379/756] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:36.246009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafecf18-74e4-4f0a-a0a9-ade8e2cb0a70 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:36.278873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:36.306752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:36.346920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad472aee-7222-4d44-b80e-501447a007dc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:36.350189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafecf18-74e4-4f0a-a0a9-ade8e2cb0a70 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:36.385395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead9a44-1509-4818-bbc0-e492cc0e89c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:36.386383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9a44-1509-4818-bbc0-e492cc0e89c7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.386868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9a44-1509-4818-bbc0-e492cc0e89c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.387290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9a44-1509-4818-bbc0-e492cc0e89c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.388333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a0a62-e12d-4bcc-a02b-0a5c6e9ea8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:36.389170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a0a62-e12d-4bcc-a02b-0a5c6e9ea8a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.389682 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a0a62-e12d-4bcc-a02b-0a5c6e9ea8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.390046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a0a62-e12d-4bcc-a02b-0a5c6e9ea8a3 None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/7d7bcb4e7c434622b650cdfc500a9680/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.392260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a0a62-e12d-4bcc-a02b-0a5c6e9ea8a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=7d7bcb4e7c434622b650cdfc500a9680, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:36.393817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a0a62-e12d-4bcc-a02b-0a5c6e9ea8a3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:36.396348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead9a44-1509-4818-bbc0-e492cc0e89c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:36.398079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead9a44-1509-4818-bbc0-e492cc0e89c7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-fead9a44-1509-4818-bbc0-e492cc0e89c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:36.410399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:36.413695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 378/757] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:36.414657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead9a44-1509-4818-bbc0-e492cc0e89c7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:36.477363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 380/758] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:53:36 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/7d7bcb4e7c434622b650cdfc500a9680/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:36.478438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a0a62-e12d-4bcc-a02b-0a5c6e9ea8a3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:36.549989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead9a44-1509-4818-bbc0-e492cc0e89c7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:36.580479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:36.596131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c7e02-32c8-4455-a683-81cc8a29e5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:36.597092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c7e02-32c8-4455-a683-81cc8a29e5ad None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.597928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c7e02-32c8-4455-a683-81cc8a29e5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.598215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c7e02-32c8-4455-a683-81cc8a29e5ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.599304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c7e02-32c8-4455-a683-81cc8a29e5ad None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=e3311c1ac5e349fa9ba83a5820171b61) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:36.599618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c7e02-32c8-4455-a683-81cc8a29e5ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:36.601981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c7e02-32c8-4455-a683-81cc8a29e5ad None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:36.610357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 379/759] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:36 2022] GET /identity/v3/role_assignments?user.id=e3311c1ac5e349fa9ba83a5820171b61&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:36.612411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b17d89-4127-4a59-a21e-a7145599cd20 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.612705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b17d89-4127-4a59-a21e-a7145599cd20 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.613007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b17d89-4127-4a59-a21e-a7145599cd20 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.620689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 380/760] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:36.626166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aee292-8b5e-4b1f-a371-e83313586f55 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.626496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aee292-8b5e-4b1f-a371-e83313586f55 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.626718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aee292-8b5e-4b1f-a371-e83313586f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.668432 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-69aee292-8b5e-4b1f-a371-e83313586f55 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:36.669755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-69aee292-8b5e-4b1f-a371-e83313586f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:36.689221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:36.701482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a0a62-e12d-4bcc-a02b-0a5c6e9ea8a3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:36.750094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fc3aa-2bb5-4bea-8e5c-31bea04ac820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:36.750638 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 381/761] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:36.750916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fc3aa-2bb5-4bea-8e5c-31bea04ac820 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.751204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fc3aa-2bb5-4bea-8e5c-31bea04ac820 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.751487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fc3aa-2bb5-4bea-8e5c-31bea04ac820 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.752312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fc3aa-2bb5-4bea-8e5c-31bea04ac820 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:36.752624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fc3aa-2bb5-4bea-8e5c-31bea04ac820 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:36.758470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fc3aa-2bb5-4bea-8e5c-31bea04ac820 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:36.776902 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aee292-8b5e-4b1f-a371-e83313586f55 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:36.791449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 381/762] 10.222.0.86 () {56 vars in 1135 bytes} [Thu Jul 28 17:53:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:36.797921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fc3aa-2bb5-4bea-8e5c-31bea04ac820 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:36.893593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aee292-8b5e-4b1f-a371-e83313586f55 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:36.913000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:36.928214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fc3aa-2bb5-4bea-8e5c-31bea04ac820 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:36.944519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea026618-6884-453c-b8cf-c1126882fd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:36.945209 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea026618-6884-453c-b8cf-c1126882fd9b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.945492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea026618-6884-453c-b8cf-c1126882fd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.945765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea026618-6884-453c-b8cf-c1126882fd9b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.946408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea026618-6884-453c-b8cf-c1126882fd9b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:36.947773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea026618-6884-453c-b8cf-c1126882fd9b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ea026618-6884-453c-b8cf-c1126882fd9b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:36.963186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:36.964264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:36.965217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 382/763] 10.222.0.86 () {56 vars in 1129 bytes} [Thu Jul 28 17:53:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:36.973332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea026618-6884-453c-b8cf-c1126882fd9b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:37.004008 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249f585-9bb1-47b9-a4b7-2cd3216f953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:37.004554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249f585-9bb1-47b9-a4b7-2cd3216f953a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:37.004840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249f585-9bb1-47b9-a4b7-2cd3216f953a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:37.005146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249f585-9bb1-47b9-a4b7-2cd3216f953a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:37.052802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:37.123419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e249f585-9bb1-47b9-a4b7-2cd3216f953a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:37.124884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e249f585-9bb1-47b9-a4b7-2cd3216f953a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:37.191019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea026618-6884-453c-b8cf-c1126882fd9b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:37.227726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 382/764] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:53:37.235991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c5e98-8d4e-4e3b-b095-c26fb3623716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:37.236937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c5e98-8d4e-4e3b-b095-c26fb3623716 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:37.237482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c5e98-8d4e-4e3b-b095-c26fb3623716 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:37.237982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249f585-9bb1-47b9-a4b7-2cd3216f953a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:37.238107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c5e98-8d4e-4e3b-b095-c26fb3623716 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:37.239009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c5e98-8d4e-4e3b-b095-c26fb3623716 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:37.239530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c5e98-8d4e-4e3b-b095-c26fb3623716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:37.241375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c5e98-8d4e-4e3b-b095-c26fb3623716 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:37.247033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 383/765] 10.222.0.86 () {56 vars in 1124 bytes} [Thu Jul 28 17:53:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:37.254133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c5e98-8d4e-4e3b-b095-c26fb3623716 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:37.291325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:37.314614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:37.344065 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249f585-9bb1-47b9-a4b7-2cd3216f953a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:37.389498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c5e98-8d4e-4e3b-b095-c26fb3623716 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:37.409005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47e533-c1ec-42ab-b4ee-fd63a6b15653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:37.410251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47e533-c1ec-42ab-b4ee-fd63a6b15653 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:37.410690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47e533-c1ec-42ab-b4ee-fd63a6b15653 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:37.411221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47e533-c1ec-42ab-b4ee-fd63a6b15653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:37.428298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47e533-c1ec-42ab-b4ee-fd63a6b15653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:37.430031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47e533-c1ec-42ab-b4ee-fd63a6b15653 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:37.431679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8991b-b181-42ee-97ff-e7015e6fc471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:37.433791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8991b-b181-42ee-97ff-e7015e6fc471 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:37.435314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8991b-b181-42ee-97ff-e7015e6fc471 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:37.436310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8991b-b181-42ee-97ff-e7015e6fc471 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:37.441833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8991b-b181-42ee-97ff-e7015e6fc471 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:37.442380 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8991b-b181-42ee-97ff-e7015e6fc471 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:37.443969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8991b-b181-42ee-97ff-e7015e6fc471 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-6b47e533-c1ec-42ab-b4ee-fd63a6b15653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:37.455289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:37.456232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:37.457252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:37.457252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:37.457252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:37.457252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:37.459250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 384/766] 10.222.0.86 () {56 vars in 1124 bytes} [Thu Jul 28 17:53:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:37.459677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 383/767] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:37.464522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8991b-b181-42ee-97ff-e7015e6fc471 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:37.469883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47e533-c1ec-42ab-b4ee-fd63a6b15653 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:37.596664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8991b-b181-42ee-97ff-e7015e6fc471 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:37.601499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47e533-c1ec-42ab-b4ee-fd63a6b15653 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:37.603042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:37.608440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:37.639185 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcd22a-011c-4d4b-b973-8a7ce9910aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:37.640720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcd22a-011c-4d4b-b973-8a7ce9910aaf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:37.641470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcd22a-011c-4d4b-b973-8a7ce9910aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:37.641928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcd22a-011c-4d4b-b973-8a7ce9910aaf None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:37.642786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcd22a-011c-4d4b-b973-8a7ce9910aaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:37.643271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcd22a-011c-4d4b-b973-8a7ce9910aaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:37.644804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcd22a-011c-4d4b-b973-8a7ce9910aaf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:37.645260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ec48c-ba64-46bf-8b36-0638b1500d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:37.645933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec48c-ba64-46bf-8b36-0638b1500d93 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:37.646218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec48c-ba64-46bf-8b36-0638b1500d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:37.646491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec48c-ba64-46bf-8b36-0638b1500d93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:37.647245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ec48c-ba64-46bf-8b36-0638b1500d93 None admin] RBAC: Adding query filter params (interface=public, service_id=56b9af6ac897404ea5c8662cc6d8eb61, region_id=RegionOne) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:37.647553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ec48c-ba64-46bf-8b36-0638b1500d93 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:37.648899 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ec48c-ba64-46bf-8b36-0638b1500d93 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:37.657065 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 384/768] 10.222.0.86 () {56 vars in 1251 bytes} [Thu Jul 28 17:53:37 2022] GET /identity/v3/endpoints?service_id=56b9af6ac897404ea5c8662cc6d8eb61&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:37.659238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283545d-9c5e-4823-b453-b36ee6a8e201 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:37.659526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283545d-9c5e-4823-b453-b36ee6a8e201 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:37.659802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283545d-9c5e-4823-b453-b36ee6a8e201 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:37.661125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 385/769] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:37.665246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 385/770] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:37.666433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b624-2342-4754-a266-ffd99add934e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:37.666716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b624-2342-4754-a266-ffd99add934e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:37.666990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b624-2342-4754-a266-ffd99add934e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:37.669616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcd22a-011c-4d4b-b973-8a7ce9910aaf None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:37.686684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-05a8b624-2342-4754-a266-ffd99add934e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:37.688210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-05a8b624-2342-4754-a266-ffd99add934e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:37.762424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 386/771] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:37.763684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8b624-2342-4754-a266-ffd99add934e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:37.819074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcd22a-011c-4d4b-b973-8a7ce9910aaf None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:37.864819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a1b37-e143-4746-b501-0c085c582c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:37.865399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1b37-e143-4746-b501-0c085c582c75 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:37.865946 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1b37-e143-4746-b501-0c085c582c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:37.866231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1b37-e143-4746-b501-0c085c582c75 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:37.866898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a1b37-e143-4746-b501-0c085c582c75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:37.868552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a1b37-e143-4746-b501-0c085c582c75 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:37.876447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 386/772] 10.222.0.86 () {56 vars in 1087 bytes} [Thu Jul 28 17:53:37 2022] GET /identity/v3/services => generated 772 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:37.877746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a1b37-e143-4746-b501-0c085c582c75 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:37.881090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:37.901655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8b624-2342-4754-a266-ffd99add934e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:37.928139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:37.992328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0e3f0-24cf-4b7e-bdf1-1a087e78f4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:37.993046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0e3f0-24cf-4b7e-bdf1-1a087e78f4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:37.993339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0e3f0-24cf-4b7e-bdf1-1a087e78f4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:37.993622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0e3f0-24cf-4b7e-bdf1-1a087e78f4a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.011007 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0e3f0-24cf-4b7e-bdf1-1a087e78f4a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:38.013195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0e3f0-24cf-4b7e-bdf1-1a087e78f4a8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-e7b0e3f0-24cf-4b7e-bdf1-1a087e78f4a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:38.034149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:38.035872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:38.036790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:38.036790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:38.036790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:38.036790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:38.036790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:38.036790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:38.036790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:38.036790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:38.039627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 387/773] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:38.045662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0e3f0-24cf-4b7e-bdf1-1a087e78f4a8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:38.049300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a1b37-e143-4746-b501-0c085c582c75 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:38.096430 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847ea4f-c260-4993-8a03-6c58f96845df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:38.097547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847ea4f-c260-4993-8a03-6c58f96845df None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.097837 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847ea4f-c260-4993-8a03-6c58f96845df None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.098118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847ea4f-c260-4993-8a03-6c58f96845df None admin] PATH_INFO: `/v3/services/network` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.098774 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847ea4f-c260-4993-8a03-6c58f96845df None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:38.100503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847ea4f-c260-4993-8a03-6c58f96845df None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-0847ea4f-c260-4993-8a03-6c58f96845df None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:38.113980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:38.115360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:38.116507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 387/774] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:37 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:38.122150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847ea4f-c260-4993-8a03-6c58f96845df None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:38.166766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0e3f0-24cf-4b7e-bdf1-1a087e78f4a8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:38.206225 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a012f1-50a4-438d-9d94-3cfebcf5200d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:38.207330 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a012f1-50a4-438d-9d94-3cfebcf5200d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.208267 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a012f1-50a4-438d-9d94-3cfebcf5200d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.208741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a012f1-50a4-438d-9d94-3cfebcf5200d None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.209785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a012f1-50a4-438d-9d94-3cfebcf5200d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:38.210302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a012f1-50a4-438d-9d94-3cfebcf5200d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:38.212151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a012f1-50a4-438d-9d94-3cfebcf5200d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:38.225801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 388/775] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:38.231658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a012f1-50a4-438d-9d94-3cfebcf5200d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:38.246214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:38.253611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847ea4f-c260-4993-8a03-6c58f96845df None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:38.286827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:38.290874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ad4e4-322c-473d-8893-8ec049150513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:38.291807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ad4e4-322c-473d-8893-8ec049150513 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.292464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ad4e4-322c-473d-8893-8ec049150513 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.293458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ad4e4-322c-473d-8893-8ec049150513 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.294411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ad4e4-322c-473d-8893-8ec049150513 None admin] RBAC: Adding query filter params (name=network) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:38.295521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ad4e4-322c-473d-8893-8ec049150513 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:38.297106 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ad4e4-322c-473d-8893-8ec049150513 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:38.303029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 388/776] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:53:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:38.308876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ad4e4-322c-473d-8893-8ec049150513 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:38.424033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ad4e4-322c-473d-8893-8ec049150513 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:38.447659 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a012f1-50a4-438d-9d94-3cfebcf5200d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:38.489387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b293e0d-f03b-4df5-8a01-a3a30c39ed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:38.490682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293e0d-f03b-4df5-8a01-a3a30c39ed2f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.490975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293e0d-f03b-4df5-8a01-a3a30c39ed2f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.491459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293e0d-f03b-4df5-8a01-a3a30c39ed2f None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.492162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b293e0d-f03b-4df5-8a01-a3a30c39ed2f None admin] RBAC: Adding query filter params (type=network) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:38.492472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b293e0d-f03b-4df5-8a01-a3a30c39ed2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:38.493879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b293e0d-f03b-4df5-8a01-a3a30c39ed2f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:38.505021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 389/777] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:53:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:38.506057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f768b-a7f2-4dd4-9be4-bc432b3d7f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:38.506683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f768b-a7f2-4dd4-9be4-bc432b3d7f2f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.506966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f768b-a7f2-4dd4-9be4-bc432b3d7f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.507290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f768b-a7f2-4dd4-9be4-bc432b3d7f2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.508098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f768b-a7f2-4dd4-9be4-bc432b3d7f2f None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=3077d844257c48b3b4ac2db9177fb6df) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:38.508411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f768b-a7f2-4dd4-9be4-bc432b3d7f2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:38.509883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f768b-a7f2-4dd4-9be4-bc432b3d7f2f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:38.512662 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b293e0d-f03b-4df5-8a01-a3a30c39ed2f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:38.525627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 389/778] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:38 2022] GET /identity/v3/role_assignments?user.id=3077d844257c48b3b4ac2db9177fb6df&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:38.549042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:38.632482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b293e0d-f03b-4df5-8a01-a3a30c39ed2f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:38.685663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef1d59-8e9f-401b-80a4-3bee5d941a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:38.686214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef1d59-8e9f-401b-80a4-3bee5d941a42 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.686503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef1d59-8e9f-401b-80a4-3bee5d941a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.686801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef1d59-8e9f-401b-80a4-3bee5d941a42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.687633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef1d59-8e9f-401b-80a4-3bee5d941a42 None admin] RBAC: Adding query filter params (interface=public, service_id=4aa7ce053e9c445b97dc93b47520e3f0, region_id=RegionOne) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:38.687956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef1d59-8e9f-401b-80a4-3bee5d941a42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:38.689546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef1d59-8e9f-401b-80a4-3bee5d941a42 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:38.698804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 390/779] 10.222.0.86 () {56 vars in 1251 bytes} [Thu Jul 28 17:53:38 2022] GET /identity/v3/endpoints?service_id=4aa7ce053e9c445b97dc93b47520e3f0&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:38.705175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f768b-a7f2-4dd4-9be4-bc432b3d7f2f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:38.825657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:38.845361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f768b-a7f2-4dd4-9be4-bc432b3d7f2f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:38.881534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba13e10-5e21-4bdc-ad3b-266446341749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:38.882163 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba13e10-5e21-4bdc-ad3b-266446341749 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.882448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba13e10-5e21-4bdc-ad3b-266446341749 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.882723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba13e10-5e21-4bdc-ad3b-266446341749 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.883391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba13e10-5e21-4bdc-ad3b-266446341749 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:38.884758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba13e10-5e21-4bdc-ad3b-266446341749 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:38.890662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 390/780] 10.222.0.86 () {56 vars in 1087 bytes} [Thu Jul 28 17:53:38 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:39.150482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f944-fed9-4bbf-a0c7-96f1f52be65f None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:39.150863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f944-fed9-4bbf-a0c7-96f1f52be65f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:39.151055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f944-fed9-4bbf-a0c7-96f1f52be65f None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:39.152435 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 391/781] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:53:39.158753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecfc7b-4671-444f-962e-474959e1e84c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:39.159057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecfc7b-4671-444f-962e-474959e1e84c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:39.159315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecfc7b-4671-444f-962e-474959e1e84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:39.179314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ff0f6-7916-41f4-8e3b-3fd6fb338df9 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:39.180089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ff0f6-7916-41f4-8e3b-3fd6fb338df9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:39.180579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ff0f6-7916-41f4-8e3b-3fd6fb338df9 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:39.181454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-43ecfc7b-4671-444f-962e-474959e1e84c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:39.182705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 392/782] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:53:39.183903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-43ecfc7b-4671-444f-962e-474959e1e84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:39.190274 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc9541-88b6-4591-ba37-e7d2e3aa914a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:39.190858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc9541-88b6-4591-ba37-e7d2e3aa914a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:39.191357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc9541-88b6-4591-ba37-e7d2e3aa914a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:39.214533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-45fc9541-88b6-4591-ba37-e7d2e3aa914a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:39.216579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-45fc9541-88b6-4591-ba37-e7d2e3aa914a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:39.303067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 391/783] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:39.312797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 393/784] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:53:39.346455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecfc7b-4671-444f-962e-474959e1e84c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:39.348507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc9541-88b6-4591-ba37-e7d2e3aa914a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:39.370241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:39.507307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:39.510390 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc9541-88b6-4591-ba37-e7d2e3aa914a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:39.554244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecfc7b-4671-444f-962e-474959e1e84c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:39.577384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d801fd-64dc-47d8-8a66-9dacdf5c5789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:39.578040 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d801fd-64dc-47d8-8a66-9dacdf5c5789 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:39.578319 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d801fd-64dc-47d8-8a66-9dacdf5c5789 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:39.578595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d801fd-64dc-47d8-8a66-9dacdf5c5789 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:39.598312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d801fd-64dc-47d8-8a66-9dacdf5c5789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-07d801fd-64dc-47d8-8a66-9dacdf5c5789 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:39.603214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:39.604148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:39.605485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 394/785] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:39.614542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d801fd-64dc-47d8-8a66-9dacdf5c5789 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:39.624201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590bc7-bedc-43c9-b746-27863f8a2f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:39.624871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590bc7-bedc-43c9-b746-27863f8a2f7e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:39.625423 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590bc7-bedc-43c9-b746-27863f8a2f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:39.625701 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590bc7-bedc-43c9-b746-27863f8a2f7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:39.646494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39590bc7-bedc-43c9-b746-27863f8a2f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-39590bc7-bedc-43c9-b746-27863f8a2f7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:39.651645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:39.652587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:39.653468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:39.662929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 392/786] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:39.663779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590bc7-bedc-43c9-b746-27863f8a2f7e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:39.763625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:39.802554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d801fd-64dc-47d8-8a66-9dacdf5c5789 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:39.847219 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590bc7-bedc-43c9-b746-27863f8a2f7e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:39.868138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856c4d1-5820-4bf0-8705-aa0daec37fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:39.868764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856c4d1-5820-4bf0-8705-aa0daec37fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:39.869068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856c4d1-5820-4bf0-8705-aa0daec37fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:39.869345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856c4d1-5820-4bf0-8705-aa0daec37fa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:39.870075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856c4d1-5820-4bf0-8705-aa0daec37fa2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:39.870381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856c4d1-5820-4bf0-8705-aa0daec37fa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:39.871753 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856c4d1-5820-4bf0-8705-aa0daec37fa2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:39.889411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 395/787] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:39.896810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856c4d1-5820-4bf0-8705-aa0daec37fa2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:39.917897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504700ff-da64-400e-b9a3-ff60d1395aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:39.918528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504700ff-da64-400e-b9a3-ff60d1395aef None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:39.918811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504700ff-da64-400e-b9a3-ff60d1395aef None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:39.919271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504700ff-da64-400e-b9a3-ff60d1395aef None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:39.920044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504700ff-da64-400e-b9a3-ff60d1395aef None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:39.921773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504700ff-da64-400e-b9a3-ff60d1395aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:39.923860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504700ff-da64-400e-b9a3-ff60d1395aef None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:39.926916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:39.941281 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 393/788] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:39.947659 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504700ff-da64-400e-b9a3-ff60d1395aef None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.050506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856c4d1-5820-4bf0-8705-aa0daec37fa2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.113793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d93fc-eb71-414e-8eaa-db729f525d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.114448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d93fc-eb71-414e-8eaa-db729f525d0a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.114731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d93fc-eb71-414e-8eaa-db729f525d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.115002 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d93fc-eb71-414e-8eaa-db729f525d0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.136993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d93fc-eb71-414e-8eaa-db729f525d0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.138446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d93fc-eb71-414e-8eaa-db729f525d0a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.143741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:40.156680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504700ff-da64-400e-b9a3-ff60d1395aef None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-0f0d93fc-eb71-414e-8eaa-db729f525d0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:40.165132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:40.167167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:40.169230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:40.169230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:40.169230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:40.169230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:40.170918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 396/789] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:40.178055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d93fc-eb71-414e-8eaa-db729f525d0a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.224079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c821e-48a2-4437-b8e3-e64690ffa8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.224781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c821e-48a2-4437-b8e3-e64690ffa8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.225095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c821e-48a2-4437-b8e3-e64690ffa8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.226467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c821e-48a2-4437-b8e3-e64690ffa8a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.249650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c821e-48a2-4437-b8e3-e64690ffa8a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.251446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c821e-48a2-4437-b8e3-e64690ffa8a8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0f9c821e-48a2-4437-b8e3-e64690ffa8a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:40.275606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:40.277778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:40.279941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:40.279941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:40.279941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:40.279941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:40.279941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 394/790] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:40.286048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c821e-48a2-4437-b8e3-e64690ffa8a8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.369905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d93fc-eb71-414e-8eaa-db729f525d0a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.380433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:40.432762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac60157-ae9b-4f96-bc56-4d230c4b07fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.433414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac60157-ae9b-4f96-bc56-4d230c4b07fb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.434423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac60157-ae9b-4f96-bc56-4d230c4b07fb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.434705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac60157-ae9b-4f96-bc56-4d230c4b07fb None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.435922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac60157-ae9b-4f96-bc56-4d230c4b07fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:40.436244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac60157-ae9b-4f96-bc56-4d230c4b07fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.437647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac60157-ae9b-4f96-bc56-4d230c4b07fb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.468182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 397/791] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:40.469106 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac60157-ae9b-4f96-bc56-4d230c4b07fb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.482853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c821e-48a2-4437-b8e3-e64690ffa8a8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.531163 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1543d6-60f7-408c-9f25-5cdb876208ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.532033 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1543d6-60f7-408c-9f25-5cdb876208ab None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.532316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1543d6-60f7-408c-9f25-5cdb876208ab None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.533473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1543d6-60f7-408c-9f25-5cdb876208ab None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.534184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1543d6-60f7-408c-9f25-5cdb876208ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:40.534493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1543d6-60f7-408c-9f25-5cdb876208ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.535865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1543d6-60f7-408c-9f25-5cdb876208ab None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.562474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 395/792] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:40.571473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1543d6-60f7-408c-9f25-5cdb876208ab None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.587482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:40.649614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac60157-ae9b-4f96-bc56-4d230c4b07fb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.684498 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1543d6-60f7-408c-9f25-5cdb876208ab None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.695845 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:40.697964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52be919-a242-4376-932c-79df96435caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.699099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52be919-a242-4376-932c-79df96435caf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.699532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52be919-a242-4376-932c-79df96435caf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.700119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52be919-a242-4376-932c-79df96435caf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.709739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52be919-a242-4376-932c-79df96435caf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.712097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52be919-a242-4376-932c-79df96435caf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.721744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4df7be-1c60-44b1-8aad-c99a4c6e402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.722395 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4df7be-1c60-44b1-8aad-c99a4c6e402f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.722684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4df7be-1c60-44b1-8aad-c99a4c6e402f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.722963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4df7be-1c60-44b1-8aad-c99a4c6e402f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-c52be919-a242-4376-932c-79df96435caf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:40.727867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:40.729105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:40.730220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:40.730220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:40.730220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:40.730220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:40.730220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:40.730220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:40.730220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:40.730220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:40.730220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:53:40.730220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:40.733281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 398/793] 10.222.0.86 () {56 vars in 1093 bytes} [Thu Jul 28 17:53:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:40.734634 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4f98-00eb-4496-b0e6-dfccbc2f3471 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.735068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4f98-00eb-4496-b0e6-dfccbc2f3471 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.735511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4f98-00eb-4496-b0e6-dfccbc2f3471 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.737448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 399/794] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:40.738370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4df7be-1c60-44b1-8aad-c99a4c6e402f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.739140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a4f98-00eb-4496-b0e6-dfccbc2f3471 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.739891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4df7be-1c60-44b1-8aad-c99a4c6e402f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-4f4df7be-1c60-44b1-8aad-c99a4c6e402f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:40.759758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:40.762525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:40.763543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:40.763543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:40.763543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:40.763543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:40.763543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:40.763543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:40.763543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:40.763543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:40.763543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:53:40.763543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:40.765221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 396/795] 10.222.0.86 () {56 vars in 1117 bytes} [Thu Jul 28 17:53:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:40.766613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046feff-758f-4c89-8ef2-7fd56f0a6dab None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.767087 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046feff-758f-4c89-8ef2-7fd56f0a6dab None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.767539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046feff-758f-4c89-8ef2-7fd56f0a6dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.789974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f046feff-758f-4c89-8ef2-7fd56f0a6dab None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:40.791188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f046feff-758f-4c89-8ef2-7fd56f0a6dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:40.878744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a4f98-00eb-4496-b0e6-dfccbc2f3471 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.879989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 397/796] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:40.881021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046feff-758f-4c89-8ef2-7fd56f0a6dab None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.898429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:40.926604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48679eaa-c82f-4777-828d-e6a9ca0f24f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.926604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48679eaa-c82f-4777-828d-e6a9ca0f24f1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.926604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48679eaa-c82f-4777-828d-e6a9ca0f24f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.926604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48679eaa-c82f-4777-828d-e6a9ca0f24f1 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.927042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48679eaa-c82f-4777-828d-e6a9ca0f24f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:40.927042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48679eaa-c82f-4777-828d-e6a9ca0f24f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.927042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48679eaa-c82f-4777-828d-e6a9ca0f24f1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.952901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 400/797] 10.222.0.86 () {56 vars in 1133 bytes} [Thu Jul 28 17:53:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:40.952901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48679eaa-c82f-4777-828d-e6a9ca0f24f1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.012875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:41.017641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046feff-758f-4c89-8ef2-7fd56f0a6dab None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.052638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6618e-1137-4932-9d3a-4569d2b480da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.053303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6618e-1137-4932-9d3a-4569d2b480da None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.053589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6618e-1137-4932-9d3a-4569d2b480da None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.053871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6618e-1137-4932-9d3a-4569d2b480da None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.054684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6618e-1137-4932-9d3a-4569d2b480da None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:41.054998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6618e-1137-4932-9d3a-4569d2b480da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.056424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6618e-1137-4932-9d3a-4569d2b480da None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.060140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48679eaa-c82f-4777-828d-e6a9ca0f24f1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.074949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 398/798] 10.222.0.86 () {56 vars in 1149 bytes} [Thu Jul 28 17:53:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:41.080647 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6618e-1137-4932-9d3a-4569d2b480da None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.096923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41d668-18cb-41fc-bb9e-6d163ffc5ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.097664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41d668-18cb-41fc-bb9e-6d163ffc5ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.097925 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41d668-18cb-41fc-bb9e-6d163ffc5ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.098200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41d668-18cb-41fc-bb9e-6d163ffc5ae8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.098843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41d668-18cb-41fc-bb9e-6d163ffc5ae8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.100203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41d668-18cb-41fc-bb9e-6d163ffc5ae8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-0b41d668-18cb-41fc-bb9e-6d163ffc5ae8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:41.109859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:41.110820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:41.111679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 401/799] 10.222.0.86 () {56 vars in 1129 bytes} [Thu Jul 28 17:53:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:41.113326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af055565-82a4-413d-8fca-b8c93c6ca84c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.113326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af055565-82a4-413d-8fca-b8c93c6ca84c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.113326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af055565-82a4-413d-8fca-b8c93c6ca84c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.117109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 402/800] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:41.117109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af055565-82a4-413d-8fca-b8c93c6ca84c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.158293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:41.192628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6618e-1137-4932-9d3a-4569d2b480da None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.228612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec679bc8-b492-4496-8d3c-2190e25b5e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.229551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec679bc8-b492-4496-8d3c-2190e25b5e5d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.229835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec679bc8-b492-4496-8d3c-2190e25b5e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.230116 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec679bc8-b492-4496-8d3c-2190e25b5e5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.238918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af055565-82a4-413d-8fca-b8c93c6ca84c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.255791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:41.274142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756bfaa-a874-475d-a666-37747637f69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.274850 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756bfaa-a874-475d-a666-37747637f69d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.275129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756bfaa-a874-475d-a666-37747637f69d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.275401 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756bfaa-a874-475d-a666-37747637f69d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.285744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756bfaa-a874-475d-a666-37747637f69d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.287195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756bfaa-a874-475d-a666-37747637f69d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.299644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec679bc8-b492-4496-8d3c-2190e25b5e5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.301116 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec679bc8-b492-4496-8d3c-2190e25b5e5d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d756bfaa-a874-475d-a666-37747637f69d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:41.301643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:41.302683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:41.303591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:41.303591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:41.303591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:41.303591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:41.303591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 403/801] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:41.303896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756bfaa-a874-475d-a666-37747637f69d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.391213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:41.400511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 399/802] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:53:41.403657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de035865-6027-4209-8fe2-9389f38f3023 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.403657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de035865-6027-4209-8fe2-9389f38f3023 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.403657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de035865-6027-4209-8fe2-9389f38f3023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.407292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756bfaa-a874-475d-a666-37747637f69d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.421637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-de035865-6027-4209-8fe2-9389f38f3023 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:41.422818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-de035865-6027-4209-8fe2-9389f38f3023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:41.442738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27931ccd-b962-48a8-a109-537c688f8c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.443297 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27931ccd-b962-48a8-a109-537c688f8c8c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.443586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27931ccd-b962-48a8-a109-537c688f8c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.443867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27931ccd-b962-48a8-a109-537c688f8c8c None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.444549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27931ccd-b962-48a8-a109-537c688f8c8c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:41.444860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27931ccd-b962-48a8-a109-537c688f8c8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.446255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27931ccd-b962-48a8-a109-537c688f8c8c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.450583 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 404/803] 10.222.0.86 () {56 vars in 1124 bytes} [Thu Jul 28 17:53:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:41.451761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd9653-feba-4f3d-b152-73402c8488d6 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.452048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd9653-feba-4f3d-b152-73402c8488d6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.452337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd9653-feba-4f3d-b152-73402c8488d6 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.453734 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 405/804] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:41.454272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd9653-feba-4f3d-b152-73402c8488d6 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.474939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:41.491781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 400/805] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:41.492442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de035865-6027-4209-8fe2-9389f38f3023 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.564778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd9653-feba-4f3d-b152-73402c8488d6 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.597648 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de035865-6027-4209-8fe2-9389f38f3023 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.598892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb1c79-d3f8-45c4-9737-e24201e9f77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.599512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb1c79-d3f8-45c4-9737-e24201e9f77d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.599794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb1c79-d3f8-45c4-9737-e24201e9f77d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.600081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb1c79-d3f8-45c4-9737-e24201e9f77d None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.600774 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb1c79-d3f8-45c4-9737-e24201e9f77d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:41.601118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb1c79-d3f8-45c4-9737-e24201e9f77d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.602513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb1c79-d3f8-45c4-9737-e24201e9f77d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.614669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 406/806] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:41.615739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb1c79-d3f8-45c4-9737-e24201e9f77d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.630246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:41.634811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84135f84-5661-4f61-bddb-6137eb3ca6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.635463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84135f84-5661-4f61-bddb-6137eb3ca6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.635749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84135f84-5661-4f61-bddb-6137eb3ca6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.636030 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84135f84-5661-4f61-bddb-6137eb3ca6b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.646378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84135f84-5661-4f61-bddb-6137eb3ca6b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.647815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84135f84-5661-4f61-bddb-6137eb3ca6b0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-84135f84-5661-4f61-bddb-6137eb3ca6b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:41.662837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:41.663782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:41.664683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:41.664683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:41.664683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:41.664683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:41.664683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 401/807] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:41.665579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c13af-72f7-43d7-8f1e-85cea6dc9ff9 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.665864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c13af-72f7-43d7-8f1e-85cea6dc9ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.666157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c13af-72f7-43d7-8f1e-85cea6dc9ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.683928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c67c13af-72f7-43d7-8f1e-85cea6dc9ff9 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:41.685165 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c67c13af-72f7-43d7-8f1e-85cea6dc9ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:41.713647 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:41.718317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb1c79-d3f8-45c4-9737-e24201e9f77d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.751864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 402/808] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:41.752686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c13af-72f7-43d7-8f1e-85cea6dc9ff9 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.753320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029898a-b732-4774-8915-d1bd1d9817cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.753942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029898a-b732-4774-8915-d1bd1d9817cf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.754232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029898a-b732-4774-8915-d1bd1d9817cf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.754508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029898a-b732-4774-8915-d1bd1d9817cf None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.755189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029898a-b732-4774-8915-d1bd1d9817cf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:41.755496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029898a-b732-4774-8915-d1bd1d9817cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.756854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029898a-b732-4774-8915-d1bd1d9817cf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.765307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 407/809] 10.222.0.86 () {56 vars in 1124 bytes} [Thu Jul 28 17:53:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:41.765307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029898a-b732-4774-8915-d1bd1d9817cf None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.839387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:41.862139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029898a-b732-4774-8915-d1bd1d9817cf None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.862601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c13af-72f7-43d7-8f1e-85cea6dc9ff9 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.901638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cacf8-56c2-4b37-82e5-07dbec3e4328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.902287 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cacf8-56c2-4b37-82e5-07dbec3e4328 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.902572 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cacf8-56c2-4b37-82e5-07dbec3e4328 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.902850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cacf8-56c2-4b37-82e5-07dbec3e4328 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.904241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29482a3-eac7-4108-8f3d-0d4ca3260d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.905486 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29482a3-eac7-4108-8f3d-0d4ca3260d8b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.905601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29482a3-eac7-4108-8f3d-0d4ca3260d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.909148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29482a3-eac7-4108-8f3d-0d4ca3260d8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.913866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cacf8-56c2-4b37-82e5-07dbec3e4328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:41.916138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29482a3-eac7-4108-8f3d-0d4ca3260d8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.918365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29482a3-eac7-4108-8f3d-0d4ca3260d8b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3e7cacf8-56c2-4b37-82e5-07dbec3e4328 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:41.918990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:41.919922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:41.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 403/810] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:41.921589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cacf8-56c2-4b37-82e5-07dbec3e4328 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d29482a3-eac7-4108-8f3d-0d4ca3260d8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:41.936346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:41.938029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:41.940047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:41.940047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:41.940047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:41.940047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:41.940047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:41.940047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:41.940047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:41.940047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:41.940047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 408/811] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:41.940047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29482a3-eac7-4108-8f3d-0d4ca3260d8b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.956835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:42.027570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cacf8-56c2-4b37-82e5-07dbec3e4328 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.036776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29482a3-eac7-4108-8f3d-0d4ca3260d8b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.061289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aafa47-c4aa-4337-b064-3356615ee413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.061906 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aafa47-c4aa-4337-b064-3356615ee413 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.062190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aafa47-c4aa-4337-b064-3356615ee413 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.062464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aafa47-c4aa-4337-b064-3356615ee413 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.063158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aafa47-c4aa-4337-b064-3356615ee413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:42.063463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aafa47-c4aa-4337-b064-3356615ee413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.064814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aafa47-c4aa-4337-b064-3356615ee413 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.071193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc3f8b-156d-4a60-bcd9-8a476a1cb141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.071841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc3f8b-156d-4a60-bcd9-8a476a1cb141 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.072131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc3f8b-156d-4a60-bcd9-8a476a1cb141 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.072409 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc3f8b-156d-4a60-bcd9-8a476a1cb141 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.073153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc3f8b-156d-4a60-bcd9-8a476a1cb141 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.074506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc3f8b-156d-4a60-bcd9-8a476a1cb141 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.076613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 404/812] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:42.077451 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aafa47-c4aa-4337-b064-3356615ee413 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.081123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:42.164732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:42.170546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 409/813] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:42.171364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc3f8b-156d-4a60-bcd9-8a476a1cb141 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.186129 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aafa47-c4aa-4337-b064-3356615ee413 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.232281 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a8ae0-8c20-4c3c-8c9d-3e417126b724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.232946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a8ae0-8c20-4c3c-8c9d-3e417126b724 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.233258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a8ae0-8c20-4c3c-8c9d-3e417126b724 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.233541 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a8ae0-8c20-4c3c-8c9d-3e417126b724 None admin] PATH_INFO: `/v3/services/network` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.234196 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a8ae0-8c20-4c3c-8c9d-3e417126b724 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.235722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a8ae0-8c20-4c3c-8c9d-3e417126b724 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-416a8ae0-8c20-4c3c-8c9d-3e417126b724 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:42.247734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:42.248866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:42.249753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 405/814] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:42.253277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a8ae0-8c20-4c3c-8c9d-3e417126b724 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.305384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc3f8b-156d-4a60-bcd9-8a476a1cb141 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.343115 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0255f-103c-4f90-a618-54d239a4db52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.343747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0255f-103c-4f90-a618-54d239a4db52 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.344028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0255f-103c-4f90-a618-54d239a4db52 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.344309 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0255f-103c-4f90-a618-54d239a4db52 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.345057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0255f-103c-4f90-a618-54d239a4db52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:42.345362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0255f-103c-4f90-a618-54d239a4db52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.346728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0255f-103c-4f90-a618-54d239a4db52 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.356929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 410/815] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:42.357803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0255f-103c-4f90-a618-54d239a4db52 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.371329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a8ae0-8c20-4c3c-8c9d-3e417126b724 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.408357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933953f-711e-426c-8c5d-ab738c91958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.408999 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933953f-711e-426c-8c5d-ab738c91958a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.409294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933953f-711e-426c-8c5d-ab738c91958a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.409573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933953f-711e-426c-8c5d-ab738c91958a None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.410520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933953f-711e-426c-8c5d-ab738c91958a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:42.410833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933953f-711e-426c-8c5d-ab738c91958a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.412257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933953f-711e-426c-8c5d-ab738c91958a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.423937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:42.429667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 406/816] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:42.430586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933953f-711e-426c-8c5d-ab738c91958a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.445153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:42.471876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0255f-103c-4f90-a618-54d239a4db52 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.507232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cde30-2c05-430c-8659-aeff3f70e323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.507975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cde30-2c05-430c-8659-aeff3f70e323 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.508262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cde30-2c05-430c-8659-aeff3f70e323 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.508557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cde30-2c05-430c-8659-aeff3f70e323 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.518162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cde30-2c05-430c-8659-aeff3f70e323 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.519673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cde30-2c05-430c-8659-aeff3f70e323 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-808cde30-2c05-430c-8659-aeff3f70e323 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:42.535283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:42.536249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:42.537188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:42.537188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:42.537188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:42.537188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:42.537188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:42.537188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:42.537188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:42.537188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:42.537188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 411/817] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:42.538602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cde30-2c05-430c-8659-aeff3f70e323 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.556729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933953f-711e-426c-8c5d-ab738c91958a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.593876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e97da-7a34-48fb-b56c-8c3ee7bb05cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.594586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e97da-7a34-48fb-b56c-8c3ee7bb05cc None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.594781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e97da-7a34-48fb-b56c-8c3ee7bb05cc None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.595055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e97da-7a34-48fb-b56c-8c3ee7bb05cc None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.595854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e97da-7a34-48fb-b56c-8c3ee7bb05cc None admin] RBAC: Adding query filter params (name=network) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:42.596166 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e97da-7a34-48fb-b56c-8c3ee7bb05cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.597677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e97da-7a34-48fb-b56c-8c3ee7bb05cc None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.602826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 407/818] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:42.603660 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e97da-7a34-48fb-b56c-8c3ee7bb05cc None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.649317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cde30-2c05-430c-8659-aeff3f70e323 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.664275 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:42.682661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:42.685292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadd24b-2d33-4ff1-9b14-ea8a8ae40a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.686480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadd24b-2d33-4ff1-9b14-ea8a8ae40a04 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.686764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadd24b-2d33-4ff1-9b14-ea8a8ae40a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.687040 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadd24b-2d33-4ff1-9b14-ea8a8ae40a04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.698094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcadd24b-2d33-4ff1-9b14-ea8a8ae40a04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.699549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcadd24b-2d33-4ff1-9b14-ea8a8ae40a04 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.713485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e97da-7a34-48fb-b56c-8c3ee7bb05cc None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-bcadd24b-2d33-4ff1-9b14-ea8a8ae40a04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:42.715343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:42.716248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:42.717689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:42.717689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:42.717689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:42.717689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:42.717689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 412/819] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:42.717689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadd24b-2d33-4ff1-9b14-ea8a8ae40a04 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.754613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afba23e-8488-4211-9ba1-39d177d5ac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.755296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afba23e-8488-4211-9ba1-39d177d5ac90 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.755580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afba23e-8488-4211-9ba1-39d177d5ac90 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.755862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afba23e-8488-4211-9ba1-39d177d5ac90 None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/e3311c1ac5e349fa9ba83a5820171b61/roles/ce68cefcc9b14645bbb5a2b411a23a44` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.772745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afba23e-8488-4211-9ba1-39d177d5ac90 None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=e3311c1ac5e349fa9ba83a5820171b61, role_id=ce68cefcc9b14645bbb5a2b411a23a44)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.774665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afba23e-8488-4211-9ba1-39d177d5ac90 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.822937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadd24b-2d33-4ff1-9b14-ea8a8ae40a04 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.856929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250a78c-8516-4f96-a55d-a8b6ed20d96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.858087 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250a78c-8516-4f96-a55d-a8b6ed20d96d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.858371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250a78c-8516-4f96-a55d-a8b6ed20d96d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.858645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250a78c-8516-4f96-a55d-a8b6ed20d96d None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.859425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250a78c-8516-4f96-a55d-a8b6ed20d96d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:42.859735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250a78c-8516-4f96-a55d-a8b6ed20d96d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.861214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250a78c-8516-4f96-a55d-a8b6ed20d96d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.881212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 408/820] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:53:42 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/e3311c1ac5e349fa9ba83a5820171b61/roles/ce68cefcc9b14645bbb5a2b411a23a44 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:42.881212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afba23e-8488-4211-9ba1-39d177d5ac90 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.886727 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 413/821] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:42.887855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250a78c-8516-4f96-a55d-a8b6ed20d96d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.920478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:42.995801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afba23e-8488-4211-9ba1-39d177d5ac90 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:43.020089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250a78c-8516-4f96-a55d-a8b6ed20d96d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:43.023283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:43.035853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2235db-eaef-45ba-b741-f4cef44d8662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:43.036827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2235db-eaef-45ba-b741-f4cef44d8662 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:43.037327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2235db-eaef-45ba-b741-f4cef44d8662 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:43.037752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2235db-eaef-45ba-b741-f4cef44d8662 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:43.038630 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2235db-eaef-45ba-b741-f4cef44d8662 None admin] RBAC: Adding query filter params (type=network) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:43.039108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2235db-eaef-45ba-b741-f4cef44d8662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:43.040530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2235db-eaef-45ba-b741-f4cef44d8662 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:43.047805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 409/822] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:43.048864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2235db-eaef-45ba-b741-f4cef44d8662 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:43.072493 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0933f1-2381-477b-8842-652d55fe66af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:43.073652 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0933f1-2381-477b-8842-652d55fe66af None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:43.074195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0933f1-2381-477b-8842-652d55fe66af None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:43.074738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0933f1-2381-477b-8842-652d55fe66af None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:43.076625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0933f1-2381-477b-8842-652d55fe66af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:43.077454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0933f1-2381-477b-8842-652d55fe66af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:43.079566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0933f1-2381-477b-8842-652d55fe66af None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:43.095499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 414/823] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:43.096701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0933f1-2381-477b-8842-652d55fe66af None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:43.170346 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2235db-eaef-45ba-b741-f4cef44d8662 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:43.198170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:43.208512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705d20f-5e4f-4099-bc6c-441664f020f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:43.209189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d20f-5e4f-4099-bc6c-441664f020f7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:43.209660 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d20f-5e4f-4099-bc6c-441664f020f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:43.209769 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d20f-5e4f-4099-bc6c-441664f020f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:43.210578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705d20f-5e4f-4099-bc6c-441664f020f7 None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=7d7bcb4e7c434622b650cdfc500a9680) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:43.216681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705d20f-5e4f-4099-bc6c-441664f020f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:43.216681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705d20f-5e4f-4099-bc6c-441664f020f7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:43.216908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0933f1-2381-477b-8842-652d55fe66af None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:43.219404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 410/824] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:43 2022] GET /identity/v3/role_assignments?user.id=7d7bcb4e7c434622b650cdfc500a9680&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:43.219753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705d20f-5e4f-4099-bc6c-441664f020f7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:43.256473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c43cff-57ea-40fb-859d-e37a441f2989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:43.257470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c43cff-57ea-40fb-859d-e37a441f2989 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:43.257942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c43cff-57ea-40fb-859d-e37a441f2989 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:43.258422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c43cff-57ea-40fb-859d-e37a441f2989 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:43.259370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c43cff-57ea-40fb-859d-e37a441f2989 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:43.261264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c43cff-57ea-40fb-859d-e37a441f2989 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:43.278730 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:43.337444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705d20f-5e4f-4099-bc6c-441664f020f7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:43.362816 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 415/825] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:43.387339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2a67e-d7fb-4f77-a2f9-0e7cc90d2395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:43.388039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2a67e-d7fb-4f77-a2f9-0e7cc90d2395 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:43.388321 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2a67e-d7fb-4f77-a2f9-0e7cc90d2395 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:43.389202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2a67e-d7fb-4f77-a2f9-0e7cc90d2395 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:43.398962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2a67e-d7fb-4f77-a2f9-0e7cc90d2395 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:43.400482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2a67e-d7fb-4f77-a2f9-0e7cc90d2395 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-77f2a67e-d7fb-4f77-a2f9-0e7cc90d2395 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:43.414927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:43.415972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:43.417429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:43.417429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:43.417429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:43.417429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:43.417429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:43.417429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:43.417429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:43.417429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:43.417429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:43.417429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:43.417429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 411/826] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:43.425034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c43cff-57ea-40fb-859d-e37a441f2989 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:43.552160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c43cff-57ea-40fb-859d-e37a441f2989 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:43.588798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a631a4a-fc41-420a-b78f-571ca76f1b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:43.589383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a631a4a-fc41-420a-b78f-571ca76f1b6b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:43.589679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a631a4a-fc41-420a-b78f-571ca76f1b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:43.589943 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a631a4a-fc41-420a-b78f-571ca76f1b6b None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:43.590753 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a631a4a-fc41-420a-b78f-571ca76f1b6b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:43.591067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a631a4a-fc41-420a-b78f-571ca76f1b6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:43.592530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a631a4a-fc41-420a-b78f-571ca76f1b6b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:43.607658 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:43.611433 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 416/827] 10.222.0.86 () {56 vars in 1135 bytes} [Thu Jul 28 17:53:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:43.616607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2a67e-d7fb-4f77-a2f9-0e7cc90d2395 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:43.730121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2a67e-d7fb-4f77-a2f9-0e7cc90d2395 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:43.737501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:43.766311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d84e0-f153-420f-bdff-dd486f16a3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:43.766966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d84e0-f153-420f-bdff-dd486f16a3ef None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:43.767254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d84e0-f153-420f-bdff-dd486f16a3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:43.767537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d84e0-f153-420f-bdff-dd486f16a3ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:43.778989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d84e0-f153-420f-bdff-dd486f16a3ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:43.781017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d84e0-f153-420f-bdff-dd486f16a3ef None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-785d84e0-f153-420f-bdff-dd486f16a3ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:43.796279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:43.797273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:43.798689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:43.798689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:43.798689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:43.798689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:43.798689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 412/828] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:43.806221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a631a4a-fc41-420a-b78f-571ca76f1b6b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:43.918882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a631a4a-fc41-420a-b78f-571ca76f1b6b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:43.958338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cf42a-18a5-4a98-9f4d-6a4a184cab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:43.959329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cf42a-18a5-4a98-9f4d-6a4a184cab6c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:43.959662 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cf42a-18a5-4a98-9f4d-6a4a184cab6c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:43.959942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cf42a-18a5-4a98-9f4d-6a4a184cab6c None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:43.960659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cf42a-18a5-4a98-9f4d-6a4a184cab6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:43.960988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cf42a-18a5-4a98-9f4d-6a4a184cab6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:43.962375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cf42a-18a5-4a98-9f4d-6a4a184cab6c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:43.974809 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 417/829] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:43.980651 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d84e0-f153-420f-bdff-dd486f16a3ef None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.090228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d84e0-f153-420f-bdff-dd486f16a3ef None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.125530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190b056-a937-43c3-b581-b8765094c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.126318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190b056-a937-43c3-b581-b8765094c4be None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.126611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190b056-a937-43c3-b581-b8765094c4be None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.126944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190b056-a937-43c3-b581-b8765094c4be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.134126 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190b056-a937-43c3-b581-b8765094c4be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.135582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190b056-a937-43c3-b581-b8765094c4be None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0190b056-a937-43c3-b581-b8765094c4be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:44.147798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:44.149900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:44.152098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:44.152098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:44.152098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:44.152098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:44.152098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:44.152098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:44.152098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:44.152098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:44.152098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 413/830] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:44.158967 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cf42a-18a5-4a98-9f4d-6a4a184cab6c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.227092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:44.268317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cf42a-18a5-4a98-9f4d-6a4a184cab6c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.303980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed84a9-418d-4c22-a3fb-2a6813011f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.304608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed84a9-418d-4c22-a3fb-2a6813011f87 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.304894 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed84a9-418d-4c22-a3fb-2a6813011f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.305205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed84a9-418d-4c22-a3fb-2a6813011f87 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.305966 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed84a9-418d-4c22-a3fb-2a6813011f87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:44.306281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed84a9-418d-4c22-a3fb-2a6813011f87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.307717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed84a9-418d-4c22-a3fb-2a6813011f87 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:44.325090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 418/831] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:44.327003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190b056-a937-43c3-b581-b8765094c4be None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.350372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:44.351945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb344199-559c-40ed-b818-387b6cadbaed None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.352364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb344199-559c-40ed-b818-387b6cadbaed None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.352649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb344199-559c-40ed-b818-387b6cadbaed None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.354802 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 419/832] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:53:44.359523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29636a-a90c-4192-91ea-eed9274976c5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.363832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29636a-a90c-4192-91ea-eed9274976c5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.363832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29636a-a90c-4192-91ea-eed9274976c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.379152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ba29636a-a90c-4192-91ea-eed9274976c5 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:44.380361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ba29636a-a90c-4192-91ea-eed9274976c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:44.439459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190b056-a937-43c3-b581-b8765094c4be None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.453015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 420/833] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:44.475296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29636a-a90c-4192-91ea-eed9274976c5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.483184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320a877-e84a-4ccb-9a1c-f60d3027b1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.483873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320a877-e84a-4ccb-9a1c-f60d3027b1f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.484245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320a877-e84a-4ccb-9a1c-f60d3027b1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.484836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320a877-e84a-4ccb-9a1c-f60d3027b1f4 None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/3077d844257c48b3b4ac2db9177fb6df/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.486692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320a877-e84a-4ccb-9a1c-f60d3027b1f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=3077d844257c48b3b4ac2db9177fb6df, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.488190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320a877-e84a-4ccb-9a1c-f60d3027b1f4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:44.530132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:44.552873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 414/834] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:53:44 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/3077d844257c48b3b4ac2db9177fb6df/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:44.589254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29636a-a90c-4192-91ea-eed9274976c5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.629663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87753c8-6478-4faf-8711-263759a54e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.630332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87753c8-6478-4faf-8711-263759a54e65 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.630613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87753c8-6478-4faf-8711-263759a54e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.630886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87753c8-6478-4faf-8711-263759a54e65 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.631560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87753c8-6478-4faf-8711-263759a54e65 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.632993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87753c8-6478-4faf-8711-263759a54e65 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e87753c8-6478-4faf-8711-263759a54e65 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:44.643107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:44.644186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:44.645047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 421/835] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:44.652669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320a877-e84a-4ccb-9a1c-f60d3027b1f4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.751733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:44.775401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320a877-e84a-4ccb-9a1c-f60d3027b1f4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.812873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ca53d-2702-43f3-884c-f40d8736c634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.814031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ca53d-2702-43f3-884c-f40d8736c634 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.814587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ca53d-2702-43f3-884c-f40d8736c634 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.815428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ca53d-2702-43f3-884c-f40d8736c634 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.816124 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ca53d-2702-43f3-884c-f40d8736c634 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:44.816431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ca53d-2702-43f3-884c-f40d8736c634 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.817831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ca53d-2702-43f3-884c-f40d8736c634 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:44.822806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 415/836] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:44.828459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87753c8-6478-4faf-8711-263759a54e65 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.937521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87753c8-6478-4faf-8711-263759a54e65 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.956063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:44.981836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b69db9-c188-4878-a5f3-25ecde524ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.984676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b69db9-c188-4878-a5f3-25ecde524ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.985191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b69db9-c188-4878-a5f3-25ecde524ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.985670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b69db9-c188-4878-a5f3-25ecde524ee1 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.986557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b69db9-c188-4878-a5f3-25ecde524ee1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:44.987112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b69db9-c188-4878-a5f3-25ecde524ee1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.988704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b69db9-c188-4878-a5f3-25ecde524ee1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:44.993996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 422/837] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:45.176307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a25e7c-60f9-4b17-a713-e01405dba397 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.176636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a25e7c-60f9-4b17-a713-e01405dba397 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.176861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a25e7c-60f9-4b17-a713-e01405dba397 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.178630 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 416/838] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:45.184258 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a08d1d-a9b5-43dd-bea5-b4cad59a5748 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.184545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a08d1d-a9b5-43dd-bea5-b4cad59a5748 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.184824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a08d1d-a9b5-43dd-bea5-b4cad59a5748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.203543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-33a08d1d-a9b5-43dd-bea5-b4cad59a5748 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:45.204725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-33a08d1d-a9b5-43dd-bea5-b4cad59a5748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:45.273845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 423/839] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:45.296462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a25e7c-60f9-4b17-a713-e01405dba397 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.404847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a25e7c-60f9-4b17-a713-e01405dba397 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:45.451474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aba1cb-d033-4624-a69f-1eefa039bfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:45.452094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aba1cb-d033-4624-a69f-1eefa039bfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.452373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aba1cb-d033-4624-a69f-1eefa039bfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.452645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aba1cb-d033-4624-a69f-1eefa039bfe3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.464310 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aba1cb-d033-4624-a69f-1eefa039bfe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-86aba1cb-d033-4624-a69f-1eefa039bfe3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:45.469832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:45.470759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:45.471608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 417/840] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:53:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:45.477196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a08d1d-a9b5-43dd-bea5-b4cad59a5748 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.577689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a08d1d-a9b5-43dd-bea5-b4cad59a5748 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:45.587549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:45.596343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361a229-3c9c-4e51-a233-7447e2469723 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.597093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361a229-3c9c-4e51-a233-7447e2469723 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.597093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361a229-3c9c-4e51-a233-7447e2469723 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.600884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 418/841] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:53:45.606957 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc43eab-8623-4273-bf75-09236e719e7e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.608105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc43eab-8623-4273-bf75-09236e719e7e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.608606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc43eab-8623-4273-bf75-09236e719e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.615816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:45.619348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c59c6-9598-4bec-8ae8-1cb7b56b9ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:45.619967 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c59c6-9598-4bec-8ae8-1cb7b56b9ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.620299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c59c6-9598-4bec-8ae8-1cb7b56b9ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.620576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c59c6-9598-4bec-8ae8-1cb7b56b9ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.621366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c59c6-9598-4bec-8ae8-1cb7b56b9ba0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:45.621717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c59c6-9598-4bec-8ae8-1cb7b56b9ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:45.623167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c59c6-9598-4bec-8ae8-1cb7b56b9ba0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:45.631427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0fc43eab-8623-4273-bf75-09236e719e7e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:45.632278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 424/842] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:53:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:45.633290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0fc43eab-8623-4273-bf75-09236e719e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:45.641102 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c59c6-9598-4bec-8ae8-1cb7b56b9ba0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.707395 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 419/843] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:53:45.729479 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc43eab-8623-4273-bf75-09236e719e7e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.760927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c59c6-9598-4bec-8ae8-1cb7b56b9ba0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:45.806908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5780f24-2872-4b68-a24c-8d29fbf46a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:45.807719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5780f24-2872-4b68-a24c-8d29fbf46a40 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.808129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5780f24-2872-4b68-a24c-8d29fbf46a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.808504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5780f24-2872-4b68-a24c-8d29fbf46a40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.820729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5780f24-2872-4b68-a24c-8d29fbf46a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:45.822941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5780f24-2872-4b68-a24c-8d29fbf46a40 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a5780f24-2872-4b68-a24c-8d29fbf46a40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:45.838563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:45.839640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:45.840725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:45.840725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:45.840725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:45.840725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:45.843574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc43eab-8623-4273-bf75-09236e719e7e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:45.847485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 425/844] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:45.848443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5780f24-2872-4b68-a24c-8d29fbf46a40 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.888501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:45.895682 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:45.900982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e27c-1844-4a73-8050-c78e145814a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:45.905005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e27c-1844-4a73-8050-c78e145814a5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.905005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e27c-1844-4a73-8050-c78e145814a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.905005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e27c-1844-4a73-8050-c78e145814a5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.905831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773e27c-1844-4a73-8050-c78e145814a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:45.909361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773e27c-1844-4a73-8050-c78e145814a5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-f773e27c-1844-4a73-8050-c78e145814a5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:45.927983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:45.929582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:45.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 420/845] 10.222.0.86 () {56 vars in 1132 bytes} [Thu Jul 28 17:53:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:45.944223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e27c-1844-4a73-8050-c78e145814a5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.983987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5780f24-2872-4b68-a24c-8d29fbf46a40 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:46.020279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943d9e6-9f3b-4e88-8c01-f90b8753abcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:46.021281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943d9e6-9f3b-4e88-8c01-f90b8753abcd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.021786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943d9e6-9f3b-4e88-8c01-f90b8753abcd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.022285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943d9e6-9f3b-4e88-8c01-f90b8753abcd None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.023182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943d9e6-9f3b-4e88-8c01-f90b8753abcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:46.023684 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943d9e6-9f3b-4e88-8c01-f90b8753abcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:46.025265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943d9e6-9f3b-4e88-8c01-f90b8753abcd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:46.042105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 426/846] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:46.043244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943d9e6-9f3b-4e88-8c01-f90b8753abcd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:46.086203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e27c-1844-4a73-8050-c78e145814a5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:46.135442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:46.136052 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289adb4-73af-4fd6-852d-c90b56e244b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:46.137824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289adb4-73af-4fd6-852d-c90b56e244b2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.138583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289adb4-73af-4fd6-852d-c90b56e244b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.139051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289adb4-73af-4fd6-852d-c90b56e244b2 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.140228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289adb4-73af-4fd6-852d-c90b56e244b2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:46.140718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289adb4-73af-4fd6-852d-c90b56e244b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:46.142344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289adb4-73af-4fd6-852d-c90b56e244b2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:46.150365 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 421/847] 10.222.0.86 () {56 vars in 1126 bytes} [Thu Jul 28 17:53:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:46.156902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289adb4-73af-4fd6-852d-c90b56e244b2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:46.161612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943d9e6-9f3b-4e88-8c01-f90b8753abcd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:46.204758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7840f0-7469-48cd-80cc-25c94143faf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:46.205470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7840f0-7469-48cd-80cc-25c94143faf6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.205755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7840f0-7469-48cd-80cc-25c94143faf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.206032 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7840f0-7469-48cd-80cc-25c94143faf6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.214447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7840f0-7469-48cd-80cc-25c94143faf6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:46.217099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7840f0-7469-48cd-80cc-25c94143faf6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-df7840f0-7469-48cd-80cc-25c94143faf6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:46.231141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:46.233749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:46.235581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:46.235581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:46.235581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:46.235581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:46.235581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:46.235581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:46.235581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:46.235581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:46.235581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:53:46.235581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:46.235581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 427/848] 10.222.0.86 () {56 vars in 1117 bytes} [Thu Jul 28 17:53:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:46.238448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7840f0-7469-48cd-80cc-25c94143faf6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:46.250766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:46.298192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289adb4-73af-4fd6-852d-c90b56e244b2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:46.332666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f4a1b-5020-4763-ac54-2c9e1d2fc3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:46.333327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f4a1b-5020-4763-ac54-2c9e1d2fc3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.333611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f4a1b-5020-4763-ac54-2c9e1d2fc3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.333890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f4a1b-5020-4763-ac54-2c9e1d2fc3f6 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.334588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f4a1b-5020-4763-ac54-2c9e1d2fc3f6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:46.334901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f4a1b-5020-4763-ac54-2c9e1d2fc3f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:46.336263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f4a1b-5020-4763-ac54-2c9e1d2fc3f6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:46.345635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 422/849] 10.222.0.86 () {56 vars in 1126 bytes} [Thu Jul 28 17:53:46 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:46.382905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7840f0-7469-48cd-80cc-25c94143faf6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:46.429756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290ca22-8990-43e9-82ae-53f42a2f273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:46.430744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290ca22-8990-43e9-82ae-53f42a2f273f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.431227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290ca22-8990-43e9-82ae-53f42a2f273f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.431690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290ca22-8990-43e9-82ae-53f42a2f273f None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.432693 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290ca22-8990-43e9-82ae-53f42a2f273f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:46.433208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290ca22-8990-43e9-82ae-53f42a2f273f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:46.434871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290ca22-8990-43e9-82ae-53f42a2f273f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:46.452436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:46.456984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 428/850] 10.222.0.86 () {56 vars in 1149 bytes} [Thu Jul 28 17:53:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:46.462677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f4a1b-5020-4763-ac54-2c9e1d2fc3f6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:46.580824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f4a1b-5020-4763-ac54-2c9e1d2fc3f6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:46.605620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:46.618602 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae4353-940d-4e39-b049-7ff0f71471fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:46.619229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae4353-940d-4e39-b049-7ff0f71471fe None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.619557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae4353-940d-4e39-b049-7ff0f71471fe None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.619836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae4353-940d-4e39-b049-7ff0f71471fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.695068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae4353-940d-4e39-b049-7ff0f71471fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:46.696530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae4353-940d-4e39-b049-7ff0f71471fe None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:46.807566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fd9b7-54b8-46e1-8048-644e4747b38d None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.807940 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fd9b7-54b8-46e1-8048-644e4747b38d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.808299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fd9b7-54b8-46e1-8048-644e4747b38d None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.809301 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 423/851] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:46 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:53:46.809759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 429/852] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:46.817898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fd9b7-54b8-46e1-8048-644e4747b38d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:46.818422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d61729-1bce-4667-ae56-aa9c7db1cd7f None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.818715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d61729-1bce-4667-ae56-aa9c7db1cd7f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.818992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d61729-1bce-4667-ae56-aa9c7db1cd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.839781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e6d61729-1bce-4667-ae56-aa9c7db1cd7f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:46.840991 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e6d61729-1bce-4667-ae56-aa9c7db1cd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:46.859119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:46.918756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 424/853] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:46.941032 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fd9b7-54b8-46e1-8048-644e4747b38d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:46.941375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d61729-1bce-4667-ae56-aa9c7db1cd7f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:46.979029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d61ab-ae1d-4e38-9f82-8749fe8fe67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:46.979969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d61ab-ae1d-4e38-9f82-8749fe8fe67e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.980446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d61ab-ae1d-4e38-9f82-8749fe8fe67e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.980871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d61ab-ae1d-4e38-9f82-8749fe8fe67e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.993726 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d61ab-ae1d-4e38-9f82-8749fe8fe67e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:46.996043 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d61ab-ae1d-4e38-9f82-8749fe8fe67e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d72d61ab-ae1d-4e38-9f82-8749fe8fe67e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:47.014125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:47.015251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:47.016369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:47.016369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:47.016369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:47.016369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:47.018299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 430/854] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:47.023901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d61ab-ae1d-4e38-9f82-8749fe8fe67e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:47.048760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d61729-1bce-4667-ae56-aa9c7db1cd7f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:47.084912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:47.087635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466bf27-549f-4e5d-b50b-bda44001d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:47.088319 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466bf27-549f-4e5d-b50b-bda44001d596 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.088613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466bf27-549f-4e5d-b50b-bda44001d596 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.088885 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466bf27-549f-4e5d-b50b-bda44001d596 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.100199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466bf27-549f-4e5d-b50b-bda44001d596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-6466bf27-549f-4e5d-b50b-bda44001d596 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:47.105399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:47.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:53:47.107271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:47.111604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 425/855] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:47.112881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58911d-08a1-46f0-bced-8d072f834e86 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.113194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58911d-08a1-46f0-bced-8d072f834e86 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.113472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58911d-08a1-46f0-bced-8d072f834e86 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.115002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 426/856] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:47.115518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58911d-08a1-46f0-bced-8d072f834e86 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:47.132418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:47.142159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d61ab-ae1d-4e38-9f82-8749fe8fe67e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:47.182431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8ed6a-da49-4bcd-af2d-4311a164260c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:47.183150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8ed6a-da49-4bcd-af2d-4311a164260c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.183619 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8ed6a-da49-4bcd-af2d-4311a164260c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.183755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8ed6a-da49-4bcd-af2d-4311a164260c None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.184383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8ed6a-da49-4bcd-af2d-4311a164260c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:47.184683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8ed6a-da49-4bcd-af2d-4311a164260c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:47.186673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8ed6a-da49-4bcd-af2d-4311a164260c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:47.204057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 431/857] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:47.205378 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951deddb-dcba-4292-b5e9-a4c2e94a7410 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.205911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951deddb-dcba-4292-b5e9-a4c2e94a7410 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.205911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951deddb-dcba-4292-b5e9-a4c2e94a7410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.224017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-951deddb-dcba-4292-b5e9-a4c2e94a7410 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:47.225236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-951deddb-dcba-4292-b5e9-a4c2e94a7410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:47.225508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58911d-08a1-46f0-bced-8d072f834e86 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:47.261609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e58b10-cece-4f7e-8927-8c25320404f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:47.262235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e58b10-cece-4f7e-8927-8c25320404f1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.262524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e58b10-cece-4f7e-8927-8c25320404f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.262803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e58b10-cece-4f7e-8927-8c25320404f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.264130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e58b10-cece-4f7e-8927-8c25320404f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:47.264442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e58b10-cece-4f7e-8927-8c25320404f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:47.265881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e58b10-cece-4f7e-8927-8c25320404f1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:47.278839 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 427/858] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:47.278839 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e58b10-cece-4f7e-8927-8c25320404f1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:47.300421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 432/859] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:47.301909 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951deddb-dcba-4292-b5e9-a4c2e94a7410 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:47.352213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:47.358130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:47.383171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e58b10-cece-4f7e-8927-8c25320404f1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:47.409029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951deddb-dcba-4292-b5e9-a4c2e94a7410 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:47.427854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed688019-db07-4311-8025-e6388a992d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:47.428650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688019-db07-4311-8025-e6388a992d24 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.428930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688019-db07-4311-8025-e6388a992d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.429243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688019-db07-4311-8025-e6388a992d24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.437064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed688019-db07-4311-8025-e6388a992d24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:47.438763 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed688019-db07-4311-8025-e6388a992d24 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:47.449135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57b8ba-f5b6-47aa-bb53-79fe6b13f953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:47.449790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57b8ba-f5b6-47aa-bb53-79fe6b13f953 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.450064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57b8ba-f5b6-47aa-bb53-79fe6b13f953 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.450338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57b8ba-f5b6-47aa-bb53-79fe6b13f953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ed688019-db07-4311-8025-e6388a992d24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:47.450924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:47.451759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:47.452654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:47.452654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:47.452654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:47.452654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:47.452654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:47.452654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:47.452654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:47.452654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:47.452654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 428/860] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:47.453183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed688019-db07-4311-8025-e6388a992d24 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:47.465844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57b8ba-f5b6-47aa-bb53-79fe6b13f953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:47.467556 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57b8ba-f5b6-47aa-bb53-79fe6b13f953 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-df57b8ba-f5b6-47aa-bb53-79fe6b13f953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:47.483095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:47.483977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:47.484877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:47.484877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:47.484877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:47.484877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:47.484877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 433/861] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:47.485601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57b8ba-f5b6-47aa-bb53-79fe6b13f953 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:47.565923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed688019-db07-4311-8025-e6388a992d24 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:47.585968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:47.593485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57b8ba-f5b6-47aa-bb53-79fe6b13f953 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:47.601998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5ef1-546d-4cba-b46f-6343c04f006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:47.602723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5ef1-546d-4cba-b46f-6343c04f006b None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.602967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5ef1-546d-4cba-b46f-6343c04f006b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.603208 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5ef1-546d-4cba-b46f-6343c04f006b None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.603939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e5ef1-546d-4cba-b46f-6343c04f006b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:47.605402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e5ef1-546d-4cba-b46f-6343c04f006b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:47.611152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:47.630758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8a80f-62d7-4a79-b2f9-ce0779e87ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:47.631392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8a80f-62d7-4a79-b2f9-ce0779e87ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.631659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8a80f-62d7-4a79-b2f9-ce0779e87ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.631934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8a80f-62d7-4a79-b2f9-ce0779e87ca0 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.632682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8a80f-62d7-4a79-b2f9-ce0779e87ca0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:47.633019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8a80f-62d7-4a79-b2f9-ce0779e87ca0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:47.634450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8a80f-62d7-4a79-b2f9-ce0779e87ca0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:47.646929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 434/862] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:47.647731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8a80f-62d7-4a79-b2f9-ce0779e87ca0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:47.674125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 429/863] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:53:47 2022] POST /identity/v3/services => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:47.675242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5ef1-546d-4cba-b46f-6343c04f006b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:47.781257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8a80f-62d7-4a79-b2f9-ce0779e87ca0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:47.834396 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc407e25-02b0-45c3-a69c-5f9cbe9b45bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:47.835477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc407e25-02b0-45c3-a69c-5f9cbe9b45bb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.835855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc407e25-02b0-45c3-a69c-5f9cbe9b45bb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.836665 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc407e25-02b0-45c3-a69c-5f9cbe9b45bb None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.837404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc407e25-02b0-45c3-a69c-5f9cbe9b45bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:47.837712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc407e25-02b0-45c3-a69c-5f9cbe9b45bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:47.839132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc407e25-02b0-45c3-a69c-5f9cbe9b45bb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:47.842198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5ef1-546d-4cba-b46f-6343c04f006b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:47.855071 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 435/864] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:47.861470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc407e25-02b0-45c3-a69c-5f9cbe9b45bb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:47.879535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:47.885791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33342f-d38d-48ec-a9a9-6d0c816eb7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:47.886216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33342f-d38d-48ec-a9a9-6d0c816eb7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:47.887400 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33342f-d38d-48ec-a9a9-6d0c816eb7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:47.887400 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33342f-d38d-48ec-a9a9-6d0c816eb7a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:47.887672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33342f-d38d-48ec-a9a9-6d0c816eb7a5 None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=e3311c1ac5e349fa9ba83a5820171b61) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:47.888087 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33342f-d38d-48ec-a9a9-6d0c816eb7a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:47.891540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33342f-d38d-48ec-a9a9-6d0c816eb7a5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:47.897244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 430/865] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:47 2022] GET /identity/v3/role_assignments?user.id=e3311c1ac5e349fa9ba83a5820171b61&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:47.993056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc407e25-02b0-45c3-a69c-5f9cbe9b45bb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:48.050151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4415e3-acd9-46b6-8058-d1b505e67ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:48.050151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4415e3-acd9-46b6-8058-d1b505e67ebf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:48.050855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4415e3-acd9-46b6-8058-d1b505e67ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:48.050855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4415e3-acd9-46b6-8058-d1b505e67ebf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:48.062117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4415e3-acd9-46b6-8058-d1b505e67ebf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:48.064419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4415e3-acd9-46b6-8058-d1b505e67ebf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-be4415e3-acd9-46b6-8058-d1b505e67ebf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:48.081234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:48.082250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:48.083201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:48.083201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:48.083201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:48.083201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:48.083201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:48.083201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:48.083201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:48.083201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:48.083201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:53:48.083201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:48.083201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 436/866] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:53:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:48.089942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33342f-d38d-48ec-a9a9-6d0c816eb7a5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:48.160009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:48.204160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33342f-d38d-48ec-a9a9-6d0c816eb7a5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:48.239764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11922739-2b4e-4b78-9b71-4a8c40e69363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:48.240394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11922739-2b4e-4b78-9b71-4a8c40e69363 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:48.240679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11922739-2b4e-4b78-9b71-4a8c40e69363 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:48.240958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11922739-2b4e-4b78-9b71-4a8c40e69363 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:48.241786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11922739-2b4e-4b78-9b71-4a8c40e69363 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:48.242097 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11922739-2b4e-4b78-9b71-4a8c40e69363 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:48.243536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11922739-2b4e-4b78-9b71-4a8c40e69363 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:48.262711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 431/867] 10.222.0.86 () {56 vars in 1135 bytes} [Thu Jul 28 17:53:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:48.268506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4415e3-acd9-46b6-8058-d1b505e67ebf None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:48.342645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:48.376401 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4415e3-acd9-46b6-8058-d1b505e67ebf None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:48.413044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bbcde-046f-43ba-a8e3-735167efcdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:48.413609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bbcde-046f-43ba-a8e3-735167efcdf0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:48.413893 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bbcde-046f-43ba-a8e3-735167efcdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:48.417152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bbcde-046f-43ba-a8e3-735167efcdf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:48.492499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bbcde-046f-43ba-a8e3-735167efcdf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:48.493991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bbcde-046f-43ba-a8e3-735167efcdf0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:48.511554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f1246-0d8d-4ae5-aac8-a1de491b7bbf None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:48.511884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f1246-0d8d-4ae5-aac8-a1de491b7bbf None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:48.512108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f1246-0d8d-4ae5-aac8-a1de491b7bbf None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:48.514025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 432/868] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:48.518701 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ab955-ece8-4e86-9158-d28263e3fbb8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:48.518985 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ab955-ece8-4e86-9158-d28263e3fbb8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:48.519266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ab955-ece8-4e86-9158-d28263e3fbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:48.538391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f61ab955-ece8-4e86-9158-d28263e3fbb8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:48.539585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f61ab955-ece8-4e86-9158-d28263e3fbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:48.572336 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:48.608732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 437/869] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:48 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:53:48.614581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bbcde-046f-43ba-a8e3-735167efcdf0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:48.633308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 433/870] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:48 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:53:48.648264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ab955-ece8-4e86-9158-d28263e3fbb8 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:48.715947 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:48.734358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bbcde-046f-43ba-a8e3-735167efcdf0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:48.777393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e4212-5b8c-431c-a4be-3d0adeca4395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:48.778427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e4212-5b8c-431c-a4be-3d0adeca4395 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:48.779471 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e4212-5b8c-431c-a4be-3d0adeca4395 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:48.779917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ab955-ece8-4e86-9158-d28263e3fbb8 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:48.780172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e4212-5b8c-431c-a4be-3d0adeca4395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:48.791917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e4212-5b8c-431c-a4be-3d0adeca4395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:48.793894 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e4212-5b8c-431c-a4be-3d0adeca4395 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-261e4212-5b8c-431c-a4be-3d0adeca4395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:48.810921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:48.811887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:48.812805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:48.812805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:48.812805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:48.812805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:48.815175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 438/871] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:48.820206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e4212-5b8c-431c-a4be-3d0adeca4395 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:48.821316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4230770-9a0f-48af-b328-16ba34bdfc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:48.821956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4230770-9a0f-48af-b328-16ba34bdfc9c None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:48.822238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4230770-9a0f-48af-b328-16ba34bdfc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:48.822513 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4230770-9a0f-48af-b328-16ba34bdfc9c None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:48.823243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4230770-9a0f-48af-b328-16ba34bdfc9c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:48.824594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4230770-9a0f-48af-b328-16ba34bdfc9c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:48.849762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:48.904503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 434/872] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:53:48 2022] POST /identity/v3/services => generated 269 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:53:48.947984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e4212-5b8c-431c-a4be-3d0adeca4395 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:48.995286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb1a66-607a-4b98-b1f0-1949a417854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:48.995803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb1a66-607a-4b98-b1f0-1949a417854a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:48.996081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb1a66-607a-4b98-b1f0-1949a417854a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:48.996381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb1a66-607a-4b98-b1f0-1949a417854a None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:48.997135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb1a66-607a-4b98-b1f0-1949a417854a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:48.997472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb1a66-607a-4b98-b1f0-1949a417854a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:48.999388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb1a66-607a-4b98-b1f0-1949a417854a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:49.013683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 439/873] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:49.019357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4230770-9a0f-48af-b328-16ba34bdfc9c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:49.131581 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4230770-9a0f-48af-b328-16ba34bdfc9c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:49.141036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:49.171592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7a6d0-bb31-4cb3-a91a-06f62b0d4292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:49.172579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7a6d0-bb31-4cb3-a91a-06f62b0d4292 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:49.173135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7a6d0-bb31-4cb3-a91a-06f62b0d4292 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:49.173617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7a6d0-bb31-4cb3-a91a-06f62b0d4292 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:49.180938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7a6d0-bb31-4cb3-a91a-06f62b0d4292 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:49.182906 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7a6d0-bb31-4cb3-a91a-06f62b0d4292 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-cda7a6d0-bb31-4cb3-a91a-06f62b0d4292 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:49.195675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:49.196609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:49.198038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:49.198038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:49.198038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:49.198038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:49.198038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:49.198038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:49.198038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:49.198038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:49.200251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 435/874] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:49.206947 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb1a66-607a-4b98-b1f0-1949a417854a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:49.292868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:49.316059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb1a66-607a-4b98-b1f0-1949a417854a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:49.351514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac902231-f879-4cf9-a894-821bc39b5c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:49.352380 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac902231-f879-4cf9-a894-821bc39b5c89 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:49.352812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac902231-f879-4cf9-a894-821bc39b5c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:49.353258 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac902231-f879-4cf9-a894-821bc39b5c89 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:49.354190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac902231-f879-4cf9-a894-821bc39b5c89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:49.354647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac902231-f879-4cf9-a894-821bc39b5c89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:49.356227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac902231-f879-4cf9-a894-821bc39b5c89 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:49.369525 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 440/875] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:49.375949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7a6d0-bb31-4cb3-a91a-06f62b0d4292 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:49.484356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7a6d0-bb31-4cb3-a91a-06f62b0d4292 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:49.518753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090eb0a-1713-415e-8109-40b5d44aa1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:49.519380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090eb0a-1713-415e-8109-40b5d44aa1bb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:49.519663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090eb0a-1713-415e-8109-40b5d44aa1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:49.519937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090eb0a-1713-415e-8109-40b5d44aa1bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:49.520738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090eb0a-1713-415e-8109-40b5d44aa1bb None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=3077d844257c48b3b4ac2db9177fb6df) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:49.521076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090eb0a-1713-415e-8109-40b5d44aa1bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:49.522516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090eb0a-1713-415e-8109-40b5d44aa1bb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:49.529410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 436/876] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:49 2022] GET /identity/v3/role_assignments?user.id=3077d844257c48b3b4ac2db9177fb6df&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:49.850708 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2f40-e247-4a29-864d-097bc2b415f8 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:49.851092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2f40-e247-4a29-864d-097bc2b415f8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:49.851307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2f40-e247-4a29-864d-097bc2b415f8 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:49.852908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 441/877] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:49.858062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1646b6-f4e5-40f3-b776-2c5a164560cf None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:49.858360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1646b6-f4e5-40f3-b776-2c5a164560cf None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:49.858641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1646b6-f4e5-40f3-b776-2c5a164560cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:49.877036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9d1646b6-f4e5-40f3-b776-2c5a164560cf None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:49.878235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9d1646b6-f4e5-40f3-b776-2c5a164560cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:49.905516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:49.947093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 437/878] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:49.970416 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d2f40-e247-4a29-864d-097bc2b415f8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:50.075104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d2f40-e247-4a29-864d-097bc2b415f8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:50.113977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f3034-c19a-433a-9538-8fc0475bb893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:50.114543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f3034-c19a-433a-9538-8fc0475bb893 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:50.114827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f3034-c19a-433a-9538-8fc0475bb893 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:50.115105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f3034-c19a-433a-9538-8fc0475bb893 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:50.115747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f3034-c19a-433a-9538-8fc0475bb893 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:50.117148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f3034-c19a-433a-9538-8fc0475bb893 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-379f3034-c19a-433a-9538-8fc0475bb893 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:50.126929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:50.128021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:50.128945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 442/879] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:50.134420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1646b6-f4e5-40f3-b776-2c5a164560cf None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:50.143187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcb261-4f95-47be-9c3a-4668f1288a3a None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:50.143498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcb261-4f95-47be-9c3a-4668f1288a3a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:50.143744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcb261-4f95-47be-9c3a-4668f1288a3a None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:50.149758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 443/880] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:50.151279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21843657-6a5b-414f-abb2-1ba1f6490b59 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:50.151572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21843657-6a5b-414f-abb2-1ba1f6490b59 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:50.151854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21843657-6a5b-414f-abb2-1ba1f6490b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:50.158827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:50.173649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-21843657-6a5b-414f-abb2-1ba1f6490b59 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:50.175665 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-21843657-6a5b-414f-abb2-1ba1f6490b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:50.254735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 444/881] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:53:50.266933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1646b6-f4e5-40f3-b776-2c5a164560cf None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:50.281839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21843657-6a5b-414f-abb2-1ba1f6490b59 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:50.318224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526362f6-8798-460b-90ca-6aea39334e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:50.318856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526362f6-8798-460b-90ca-6aea39334e5d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:50.319143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526362f6-8798-460b-90ca-6aea39334e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:50.319421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526362f6-8798-460b-90ca-6aea39334e5d None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:50.320121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526362f6-8798-460b-90ca-6aea39334e5d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:50.320432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526362f6-8798-460b-90ca-6aea39334e5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:50.322001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526362f6-8798-460b-90ca-6aea39334e5d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:50.330350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 438/882] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:50.335159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526362f6-8798-460b-90ca-6aea39334e5d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:50.369152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:50.398605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21843657-6a5b-414f-abb2-1ba1f6490b59 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:50.430939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:50.436937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd5608-b631-42c9-9aa6-9fcd07e3b2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:50.437681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd5608-b631-42c9-9aa6-9fcd07e3b2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:50.438029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd5608-b631-42c9-9aa6-9fcd07e3b2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:50.438321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd5608-b631-42c9-9aa6-9fcd07e3b2d4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:50.449384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd5608-b631-42c9-9aa6-9fcd07e3b2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-afbd5608-b631-42c9-9aa6-9fcd07e3b2d4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:50.454999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:50.455960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:50.457184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 445/883] 10.222.0.86 () {56 vars in 1120 bytes} [Thu Jul 28 17:53:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:50.470308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd5608-b631-42c9-9aa6-9fcd07e3b2d4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:50.470473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526362f6-8798-460b-90ca-6aea39334e5d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:50.508427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8436c-0d9a-4e74-baff-b16bb36c97ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:50.509319 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8436c-0d9a-4e74-baff-b16bb36c97ce None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:50.510491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8436c-0d9a-4e74-baff-b16bb36c97ce None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:50.510776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8436c-0d9a-4e74-baff-b16bb36c97ce None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:50.511530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8436c-0d9a-4e74-baff-b16bb36c97ce None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:50.511843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8436c-0d9a-4e74-baff-b16bb36c97ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:50.513272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8436c-0d9a-4e74-baff-b16bb36c97ce None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:50.519488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 439/884] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:50.525119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8436c-0d9a-4e74-baff-b16bb36c97ce None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:50.592560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd5608-b631-42c9-9aa6-9fcd07e3b2d4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:50.638742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdabb1-4f99-4d44-a498-dc2418d0b822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:50.639813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdabb1-4f99-4d44-a498-dc2418d0b822 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:50.640096 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdabb1-4f99-4d44-a498-dc2418d0b822 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:50.640521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdabb1-4f99-4d44-a498-dc2418d0b822 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:50.641487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdabb1-4f99-4d44-a498-dc2418d0b822 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:50.642192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:50.642527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdabb1-4f99-4d44-a498-dc2418d0b822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:50.644987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdabb1-4f99-4d44-a498-dc2418d0b822 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:50.652536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8436c-0d9a-4e74-baff-b16bb36c97ce None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:50.662487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 446/885] 10.222.0.86 () {56 vars in 1115 bytes} [Thu Jul 28 17:53:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:50.664091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdabb1-4f99-4d44-a498-dc2418d0b822 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:50.695452 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a55223-38f2-4a5c-ae11-9c129343841c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:50.696551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a55223-38f2-4a5c-ae11-9c129343841c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:50.696551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a55223-38f2-4a5c-ae11-9c129343841c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:50.697304 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a55223-38f2-4a5c-ae11-9c129343841c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:50.698279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a55223-38f2-4a5c-ae11-9c129343841c None admin] RBAC: Adding query filter params (interface=public, service_id=d864fb8f7acd4ceba1c2ca666dfd67af, region_id=RegionOne) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:50.698709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a55223-38f2-4a5c-ae11-9c129343841c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:50.700245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a55223-38f2-4a5c-ae11-9c129343841c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:50.711800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 440/886] 10.222.0.86 () {56 vars in 1251 bytes} [Thu Jul 28 17:53:50 2022] GET /identity/v3/endpoints?service_id=d864fb8f7acd4ceba1c2ca666dfd67af&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:50.718339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a55223-38f2-4a5c-ae11-9c129343841c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:50.749308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:50.810787 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdabb1-4f99-4d44-a498-dc2418d0b822 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:50.828481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a55223-38f2-4a5c-ae11-9c129343841c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:50.848524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd113963-25d3-46d9-8899-25f48b0a00f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:50.849436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd113963-25d3-46d9-8899-25f48b0a00f4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:50.849866 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd113963-25d3-46d9-8899-25f48b0a00f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:50.850391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd113963-25d3-46d9-8899-25f48b0a00f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:50.862393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd113963-25d3-46d9-8899-25f48b0a00f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:50.864252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd113963-25d3-46d9-8899-25f48b0a00f4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:50.864882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b06c8b-2d39-4190-8deb-1f20bde5b814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:50.865557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b06c8b-2d39-4190-8deb-1f20bde5b814 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:50.865841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b06c8b-2d39-4190-8deb-1f20bde5b814 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:50.866116 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b06c8b-2d39-4190-8deb-1f20bde5b814 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:50.866928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b06c8b-2d39-4190-8deb-1f20bde5b814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:50.869906 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b06c8b-2d39-4190-8deb-1f20bde5b814 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:50.885001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 441/887] 10.222.0.86 () {56 vars in 1087 bytes} [Thu Jul 28 17:53:50 2022] GET /identity/v3/services => generated 1268 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-dd113963-25d3-46d9-8899-25f48b0a00f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:50.886508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:50.887384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:50.888956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:50.888956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:50.888956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:50.888956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:50.888956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 447/888] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:50.898152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b06c8b-2d39-4190-8deb-1f20bde5b814 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:50.919498 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:51.015720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b06c8b-2d39-4190-8deb-1f20bde5b814 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:51.059122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404a013-eff9-4423-8bd9-bebf3f10d5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:51.060228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404a013-eff9-4423-8bd9-bebf3f10d5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.060510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404a013-eff9-4423-8bd9-bebf3f10d5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.060786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404a013-eff9-4423-8bd9-bebf3f10d5d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.061805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404a013-eff9-4423-8bd9-bebf3f10d5d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:51.061805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404a013-eff9-4423-8bd9-bebf3f10d5d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:51.065010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404a013-eff9-4423-8bd9-bebf3f10d5d6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:51.077232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 442/889] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:51.083083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd113963-25d3-46d9-8899-25f48b0a00f4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:51.092553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40195df-2522-4abe-a71d-65dec6238199 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.092832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40195df-2522-4abe-a71d-65dec6238199 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.093135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40195df-2522-4abe-a71d-65dec6238199 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.094760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 443/890] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:53:51.100929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e94091-025d-4585-9f86-cd5b80f5878f None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.101266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e94091-025d-4585-9f86-cd5b80f5878f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.108879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e94091-025d-4585-9f86-cd5b80f5878f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.121780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-77e94091-025d-4585-9f86-cd5b80f5878f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:51.129132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-77e94091-025d-4585-9f86-cd5b80f5878f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:51.196874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 444/891] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:53:51.219748 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd113963-25d3-46d9-8899-25f48b0a00f4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:51.220259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e94091-025d-4585-9f86-cd5b80f5878f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:51.228011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:51.233334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:51.268486 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19376e6-3359-4d98-9d22-c1b9619d2531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:51.269147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19376e6-3359-4d98-9d22-c1b9619d2531 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.269435 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19376e6-3359-4d98-9d22-c1b9619d2531 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.270038 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19376e6-3359-4d98-9d22-c1b9619d2531 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.280391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19376e6-3359-4d98-9d22-c1b9619d2531 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:51.282224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19376e6-3359-4d98-9d22-c1b9619d2531 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a19376e6-3359-4d98-9d22-c1b9619d2531 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:51.299456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:51.300458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:51.301445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:51.301445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:51.301445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:51.301445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:51.301445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:51.301445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:51.301445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:51.301445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:51.301445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:53:51.301445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:51.305097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 448/892] 10.222.0.86 () {56 vars in 1117 bytes} [Thu Jul 28 17:53:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:51.307827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19376e6-3359-4d98-9d22-c1b9619d2531 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:51.349375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e94091-025d-4585-9f86-cd5b80f5878f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:51.389574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430189a-30c4-44e9-8d17-862a65c1a81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:51.391270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430189a-30c4-44e9-8d17-862a65c1a81e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.391794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430189a-30c4-44e9-8d17-862a65c1a81e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.392526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430189a-30c4-44e9-8d17-862a65c1a81e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.393919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430189a-30c4-44e9-8d17-862a65c1a81e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:51.395983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430189a-30c4-44e9-8d17-862a65c1a81e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-9430189a-30c4-44e9-8d17-862a65c1a81e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:51.410347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:51.411351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:53:51.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:51.415118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 445/893] 10.222.0.86 () {56 vars in 1132 bytes} [Thu Jul 28 17:53:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:51.426410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430189a-30c4-44e9-8d17-862a65c1a81e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:51.445206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19376e6-3359-4d98-9d22-c1b9619d2531 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:51.481368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55412fa6-1de7-47bc-9a82-4b0411516061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:51.482907 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55412fa6-1de7-47bc-9a82-4b0411516061 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.483412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55412fa6-1de7-47bc-9a82-4b0411516061 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.483951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55412fa6-1de7-47bc-9a82-4b0411516061 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.485051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55412fa6-1de7-47bc-9a82-4b0411516061 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:51.485577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55412fa6-1de7-47bc-9a82-4b0411516061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:51.487306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55412fa6-1de7-47bc-9a82-4b0411516061 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:51.505948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 449/894] 10.222.0.86 () {56 vars in 1149 bytes} [Thu Jul 28 17:53:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:51.511919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55412fa6-1de7-47bc-9a82-4b0411516061 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:51.512337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:51.542466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:51.548256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430189a-30c4-44e9-8d17-862a65c1a81e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:51.584692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e32efc-0ee3-458b-882d-87d7e936549e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:51.585351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e32efc-0ee3-458b-882d-87d7e936549e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.585637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e32efc-0ee3-458b-882d-87d7e936549e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.585917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e32efc-0ee3-458b-882d-87d7e936549e None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.586604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e32efc-0ee3-458b-882d-87d7e936549e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:51.586902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e32efc-0ee3-458b-882d-87d7e936549e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:51.588429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e32efc-0ee3-458b-882d-87d7e936549e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:51.598875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 446/895] 10.222.0.86 () {56 vars in 1126 bytes} [Thu Jul 28 17:53:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:51.600186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e32efc-0ee3-458b-882d-87d7e936549e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:51.637103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55412fa6-1de7-47bc-9a82-4b0411516061 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:51.682702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f56f33-82ec-403f-9baa-8608df3e5e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:51.683496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f56f33-82ec-403f-9baa-8608df3e5e40 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.683875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f56f33-82ec-403f-9baa-8608df3e5e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.684255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f56f33-82ec-403f-9baa-8608df3e5e40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.727155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e32efc-0ee3-458b-882d-87d7e936549e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:51.758369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f56f33-82ec-403f-9baa-8608df3e5e40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:51.759746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f56f33-82ec-403f-9baa-8608df3e5e40 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:51.763421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12435ae2-9265-4586-8252-44795bdd1962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:51.764045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12435ae2-9265-4586-8252-44795bdd1962 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.764328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12435ae2-9265-4586-8252-44795bdd1962 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.764606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12435ae2-9265-4586-8252-44795bdd1962 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.765327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12435ae2-9265-4586-8252-44795bdd1962 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:51.765637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12435ae2-9265-4586-8252-44795bdd1962 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:51.766988 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12435ae2-9265-4586-8252-44795bdd1962 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:51.769040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:51.772226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 447/896] 10.222.0.86 () {56 vars in 1126 bytes} [Thu Jul 28 17:53:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:51.773462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77877902-3e2f-44db-8912-9e9bd1db4da0 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.773883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77877902-3e2f-44db-8912-9e9bd1db4da0 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.774261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77877902-3e2f-44db-8912-9e9bd1db4da0 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.780614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 448/897] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:51.781511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77877902-3e2f-44db-8912-9e9bd1db4da0 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:51.787734 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:51.865687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 450/898] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:53:51 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:53:51.867187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce0ec-aeab-4bec-8878-fdcce5e5260c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.867727 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce0ec-aeab-4bec-8878-fdcce5e5260c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.868012 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce0ec-aeab-4bec-8878-fdcce5e5260c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.886364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e37ce0ec-aeab-4bec-8878-fdcce5e5260c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:51.887592 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e37ce0ec-aeab-4bec-8878-fdcce5e5260c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:51.900389 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77877902-3e2f-44db-8912-9e9bd1db4da0 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:51.936504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b92008-d774-4fa0-9085-bc3cd51a518a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:51.937185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b92008-d774-4fa0-9085-bc3cd51a518a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.937447 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b92008-d774-4fa0-9085-bc3cd51a518a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.937722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b92008-d774-4fa0-9085-bc3cd51a518a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.938494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b92008-d774-4fa0-9085-bc3cd51a518a None admin] RBAC: Adding query filter params (interface=public, service_id=66b3188c6510497083594cd719d125d2, region_id=RegionOne) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:51.938815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b92008-d774-4fa0-9085-bc3cd51a518a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:51.940185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b92008-d774-4fa0-9085-bc3cd51a518a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:51.949775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 449/899] 10.222.0.86 () {56 vars in 1251 bytes} [Thu Jul 28 17:53:51 2022] GET /identity/v3/endpoints?service_id=66b3188c6510497083594cd719d125d2&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:51.950664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b92008-d774-4fa0-9085-bc3cd51a518a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:51.961026 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 451/900] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:51.961026 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce0ec-aeab-4bec-8878-fdcce5e5260c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:52.020062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:52.029847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:52.067605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce0ec-aeab-4bec-8878-fdcce5e5260c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:52.069100 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b92008-d774-4fa0-9085-bc3cd51a518a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:52.104212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5cfec-c5b2-4273-a90a-290348087fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:52.104845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5cfec-c5b2-4273-a90a-290348087fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:52.105156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5cfec-c5b2-4273-a90a-290348087fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:52.105472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5cfec-c5b2-4273-a90a-290348087fb2 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:52.106088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5cfec-c5b2-4273-a90a-290348087fb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:52.108004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5cfec-c5b2-4273-a90a-290348087fb2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:52.108477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbdcb1-cad5-4a45-9677-5b6158a3dc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:52.109158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbdcb1-cad5-4a45-9677-5b6158a3dc4b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:52.109444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbdcb1-cad5-4a45-9677-5b6158a3dc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:52.109726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbdcb1-cad5-4a45-9677-5b6158a3dc4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:52.113974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 452/901] 10.222.0.86 () {56 vars in 1087 bytes} [Thu Jul 28 17:53:51 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:52.114971 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5cfec-c5b2-4273-a90a-290348087fb2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:52.132058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbdcb1-cad5-4a45-9677-5b6158a3dc4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:52.134002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbdcb1-cad5-4a45-9677-5b6158a3dc4b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c4dbdcb1-cad5-4a45-9677-5b6158a3dc4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:52.150621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:52.151608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:52.152494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:52.152494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:52.152494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:52.152494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:52.155583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 450/902] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:52.161177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbdcb1-cad5-4a45-9677-5b6158a3dc4b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:52.260990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5cfec-c5b2-4273-a90a-290348087fb2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:52.287817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbdcb1-cad5-4a45-9677-5b6158a3dc4b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:52.290522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:52.304097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda32300-ad29-4087-8907-7448ea154593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:52.304804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda32300-ad29-4087-8907-7448ea154593 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:52.305079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda32300-ad29-4087-8907-7448ea154593 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:52.305352 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda32300-ad29-4087-8907-7448ea154593 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:52.306551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda32300-ad29-4087-8907-7448ea154593 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:52.307341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda32300-ad29-4087-8907-7448ea154593 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:52.309429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-bda32300-ad29-4087-8907-7448ea154593 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:52.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:52.325729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:52.326777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 453/903] 10.222.0.86 () {56 vars in 1126 bytes} [Thu Jul 28 17:53:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:52.327781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda32300-ad29-4087-8907-7448ea154593 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:52.334412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf868b-1ddb-4207-ac6e-700c4e2a9299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:52.335046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf868b-1ddb-4207-ac6e-700c4e2a9299 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:52.335333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf868b-1ddb-4207-ac6e-700c4e2a9299 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:52.335610 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf868b-1ddb-4207-ac6e-700c4e2a9299 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:52.336883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf868b-1ddb-4207-ac6e-700c4e2a9299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:52.336883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf868b-1ddb-4207-ac6e-700c4e2a9299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:52.338098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf868b-1ddb-4207-ac6e-700c4e2a9299 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:52.352060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 451/904] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:52.358427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf868b-1ddb-4207-ac6e-700c4e2a9299 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:52.447294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda32300-ad29-4087-8907-7448ea154593 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:52.492270 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe5b7b-331f-4c98-ba55-6771d7991597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:52.492894 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe5b7b-331f-4c98-ba55-6771d7991597 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:52.493203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe5b7b-331f-4c98-ba55-6771d7991597 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:52.493479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe5b7b-331f-4c98-ba55-6771d7991597 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:52.493886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf868b-1ddb-4207-ac6e-700c4e2a9299 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:52.494161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe5b7b-331f-4c98-ba55-6771d7991597 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:52.494465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe5b7b-331f-4c98-ba55-6771d7991597 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:52.495826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe5b7b-331f-4c98-ba55-6771d7991597 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:52.507517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 454/905] 10.222.0.86 () {56 vars in 1122 bytes} [Thu Jul 28 17:53:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:52.510033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe5b7b-331f-4c98-ba55-6771d7991597 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:52.541254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cc423-6879-4b35-ac9f-494a6ba5b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:52.541823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cc423-6879-4b35-ac9f-494a6ba5b4dd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:52.542101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cc423-6879-4b35-ac9f-494a6ba5b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:52.542374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cc423-6879-4b35-ac9f-494a6ba5b4dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:52.553282 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cc423-6879-4b35-ac9f-494a6ba5b4dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:52.554737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cc423-6879-4b35-ac9f-494a6ba5b4dd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:52.565759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c48cc423-6879-4b35-ac9f-494a6ba5b4dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:52.570084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:52.570979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:52.571898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:52.571898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:52.571898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:52.571898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:52.571898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:52.571898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:52.571898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:52.571898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:52.572338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 452/906] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:52.577779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cc423-6879-4b35-ac9f-494a6ba5b4dd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:52.629633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:52.633828 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe5b7b-331f-4c98-ba55-6771d7991597 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:52.677351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bab7e-99e0-430d-b41a-8c51dff28b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:52.678373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bab7e-99e0-430d-b41a-8c51dff28b4e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:52.678865 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bab7e-99e0-430d-b41a-8c51dff28b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:52.679347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bab7e-99e0-430d-b41a-8c51dff28b4e None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:52.680359 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bab7e-99e0-430d-b41a-8c51dff28b4e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:52.680928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bab7e-99e0-430d-b41a-8c51dff28b4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:52.682652 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bab7e-99e0-430d-b41a-8c51dff28b4e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:52.693596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 455/907] 10.222.0.86 () {56 vars in 1122 bytes} [Thu Jul 28 17:53:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:52.708685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cc423-6879-4b35-ac9f-494a6ba5b4dd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:52.747858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803a3f7-2c53-4ee6-9ef2-f3d50a943a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:52.748508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803a3f7-2c53-4ee6-9ef2-f3d50a943a64 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:52.748795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803a3f7-2c53-4ee6-9ef2-f3d50a943a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:52.749105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803a3f7-2c53-4ee6-9ef2-f3d50a943a64 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:52.750144 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803a3f7-2c53-4ee6-9ef2-f3d50a943a64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:52.750453 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803a3f7-2c53-4ee6-9ef2-f3d50a943a64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:52.751924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803a3f7-2c53-4ee6-9ef2-f3d50a943a64 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:52.766190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 453/908] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:52.772391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bab7e-99e0-430d-b41a-8c51dff28b4e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:52.891053 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bab7e-99e0-430d-b41a-8c51dff28b4e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:52.903001 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:52.928276 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0632f-e3ac-43b1-b432-7478686af28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:52.929360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0632f-e3ac-43b1-b432-7478686af28b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:52.929854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0632f-e3ac-43b1-b432-7478686af28b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:52.930338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0632f-e3ac-43b1-b432-7478686af28b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:52.931340 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0632f-e3ac-43b1-b432-7478686af28b None admin] RBAC: Adding query filter params (role.id=8e5389a3e81c4e58b716a69b7f74257b, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=e3311c1ac5e349fa9ba83a5820171b61) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:52.931934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0632f-e3ac-43b1-b432-7478686af28b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:52.933703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0632f-e3ac-43b1-b432-7478686af28b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:52.940697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 456/909] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:53:52 2022] GET /identity/v3/role_assignments?user.id=e3311c1ac5e349fa9ba83a5820171b61&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=8e5389a3e81c4e58b716a69b7f74257b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:53.016955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cf88-9898-46a8-94b8-ea7a171f506c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:53.017503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cf88-9898-46a8-94b8-ea7a171f506c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:53.017583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cf88-9898-46a8-94b8-ea7a171f506c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:53.019280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 454/910] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:53.025005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fc6e5-311f-443d-a16d-9f72d3a82559 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:53.025318 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fc6e5-311f-443d-a16d-9f72d3a82559 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:53.025576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fc6e5-311f-443d-a16d-9f72d3a82559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:53.044700 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-dc7fc6e5-311f-443d-a16d-9f72d3a82559 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:53.045970 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dc7fc6e5-311f-443d-a16d-9f72d3a82559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:53.119885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 457/911] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:53.143443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6cf88-9898-46a8-94b8-ea7a171f506c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:53.241312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:53.248765 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6cf88-9898-46a8-94b8-ea7a171f506c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:53.285676 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282ed54-dd2f-41fd-9e4b-e0cd6134ce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:53.286330 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282ed54-dd2f-41fd-9e4b-e0cd6134ce1f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:53.286612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282ed54-dd2f-41fd-9e4b-e0cd6134ce1f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:53.286888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282ed54-dd2f-41fd-9e4b-e0cd6134ce1f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:53.287526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282ed54-dd2f-41fd-9e4b-e0cd6134ce1f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:53.288891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282ed54-dd2f-41fd-9e4b-e0cd6134ce1f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-5282ed54-dd2f-41fd-9e4b-e0cd6134ce1f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:53.298816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:53.299817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:53.300664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 455/912] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:53.306007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fc6e5-311f-443d-a16d-9f72d3a82559 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:53.411551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fc6e5-311f-443d-a16d-9f72d3a82559 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:53.414297 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:53.448731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdfb21-39d7-45be-a3de-ff5d18970a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:53.449758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdfb21-39d7-45be-a3de-ff5d18970a2c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:53.450273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdfb21-39d7-45be-a3de-ff5d18970a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:53.450776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdfb21-39d7-45be-a3de-ff5d18970a2c None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:53.452596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdfb21-39d7-45be-a3de-ff5d18970a2c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:53.453218 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdfb21-39d7-45be-a3de-ff5d18970a2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:53.454789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdfb21-39d7-45be-a3de-ff5d18970a2c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:53.460308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 458/913] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:53.468366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282ed54-dd2f-41fd-9e4b-e0cd6134ce1f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:53.574159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282ed54-dd2f-41fd-9e4b-e0cd6134ce1f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:53.609839 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f37d00-cec5-4ec4-8406-fb984667e8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:53.610837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f37d00-cec5-4ec4-8406-fb984667e8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:53.611335 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f37d00-cec5-4ec4-8406-fb984667e8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:53.611813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f37d00-cec5-4ec4-8406-fb984667e8b7 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:53.612694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f37d00-cec5-4ec4-8406-fb984667e8b7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:53.613244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f37d00-cec5-4ec4-8406-fb984667e8b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:53.614807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f37d00-cec5-4ec4-8406-fb984667e8b7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:53.620408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 456/914] 10.222.0.86 () {56 vars in 1114 bytes} [Thu Jul 28 17:53:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:53.626617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdfb21-39d7-45be-a3de-ff5d18970a2c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:53.740152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdfb21-39d7-45be-a3de-ff5d18970a2c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:53.774697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966f452-f6ad-4cc0-90e1-d705dbce7cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:53.775357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966f452-f6ad-4cc0-90e1-d705dbce7cf7 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:53.775644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966f452-f6ad-4cc0-90e1-d705dbce7cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:53.775918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966f452-f6ad-4cc0-90e1-d705dbce7cf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:53.776644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966f452-f6ad-4cc0-90e1-d705dbce7cf7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:53.778048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966f452-f6ad-4cc0-90e1-d705dbce7cf7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:53.875241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 459/915] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:54.219001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4af60-da30-4ba4-9e93-d30a97339a9f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:54.219554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4af60-da30-4ba4-9e93-d30a97339a9f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:54.219554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4af60-da30-4ba4-9e93-d30a97339a9f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:54.221210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 457/916] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:54.226325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf6215-daf4-4c68-aed9-5f43aad22de1 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:54.226609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf6215-daf4-4c68-aed9-5f43aad22de1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:54.226884 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf6215-daf4-4c68-aed9-5f43aad22de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:54.245383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-84bf6215-daf4-4c68-aed9-5f43aad22de1 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:54.246566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-84bf6215-daf4-4c68-aed9-5f43aad22de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:54.264876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:54.325082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 460/917] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:54.343810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4af60-da30-4ba4-9e93-d30a97339a9f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:54.371006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:54.456559 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4af60-da30-4ba4-9e93-d30a97339a9f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:54.492146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94db9b-c771-4380-8716-f8898cb42935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:54.492706 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94db9b-c771-4380-8716-f8898cb42935 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:54.493009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94db9b-c771-4380-8716-f8898cb42935 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:54.493293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94db9b-c771-4380-8716-f8898cb42935 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:54.493926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94db9b-c771-4380-8716-f8898cb42935 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:54.495284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94db9b-c771-4380-8716-f8898cb42935 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-1a94db9b-c771-4380-8716-f8898cb42935 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:54.504529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:54.505575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:53:54.506629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:54.510778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 458/918] 10.222.0.86 () {56 vars in 1132 bytes} [Thu Jul 28 17:53:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:54.512129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf6215-daf4-4c68-aed9-5f43aad22de1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:54.615956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf6215-daf4-4c68-aed9-5f43aad22de1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:54.650805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbea61-cbcb-48b8-a77e-7a626d5c72bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:54.651427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbea61-cbcb-48b8-a77e-7a626d5c72bd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:54.651704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbea61-cbcb-48b8-a77e-7a626d5c72bd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:54.651974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbea61-cbcb-48b8-a77e-7a626d5c72bd None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:54.652718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbea61-cbcb-48b8-a77e-7a626d5c72bd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:54.653050 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbea61-cbcb-48b8-a77e-7a626d5c72bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:54.654985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbea61-cbcb-48b8-a77e-7a626d5c72bd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:54.664591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 461/919] 10.222.0.86 () {56 vars in 1126 bytes} [Thu Jul 28 17:53:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:54.665953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94db9b-c771-4380-8716-f8898cb42935 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:54.767530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:54.780587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94db9b-c771-4380-8716-f8898cb42935 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:54.824320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602f9d1-c700-49ae-a426-c3c2e7c94c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:54.824951 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602f9d1-c700-49ae-a426-c3c2e7c94c74 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:54.825288 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602f9d1-c700-49ae-a426-c3c2e7c94c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:54.826089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602f9d1-c700-49ae-a426-c3c2e7c94c74 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:54.826776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602f9d1-c700-49ae-a426-c3c2e7c94c74 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:54.827081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602f9d1-c700-49ae-a426-c3c2e7c94c74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:54.828482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602f9d1-c700-49ae-a426-c3c2e7c94c74 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:54.838605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 459/920] 10.222.0.86 () {56 vars in 1126 bytes} [Thu Jul 28 17:53:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:54.839996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbea61-cbcb-48b8-a77e-7a626d5c72bd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:54.860853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:54.948000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbea61-cbcb-48b8-a77e-7a626d5c72bd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:54.979867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee971a-398a-4187-81d6-fc5272ae3a69 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:54.980326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee971a-398a-4187-81d6-fc5272ae3a69 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:54.980712 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee971a-398a-4187-81d6-fc5272ae3a69 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:54.982494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 460/921] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:53:54.984686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee818f-a556-4c0e-87ab-abf9d25f555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:54.985502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee818f-a556-4c0e-87ab-abf9d25f555e None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:54.985931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee818f-a556-4c0e-87ab-abf9d25f555e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:54.986425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee818f-a556-4c0e-87ab-abf9d25f555e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:54.987300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee818f-a556-4c0e-87ab-abf9d25f555e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:54.987917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114a68f-dc39-4d29-8948-c0d50ae33ba6 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:54.988325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114a68f-dc39-4d29-8948-c0d50ae33ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:54.988704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114a68f-dc39-4d29-8948-c0d50ae33ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:54.989397 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee818f-a556-4c0e-87ab-abf9d25f555e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:55.008726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e114a68f-dc39-4d29-8948-c0d50ae33ba6 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:55.010081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e114a68f-dc39-4d29-8948-c0d50ae33ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:55.078226 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 462/922] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:55.105966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 461/923] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:53:55.112463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0557d1a-9643-456c-9b64-920456a621c7 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:55.112750 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0557d1a-9643-456c-9b64-920456a621c7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:55.113065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0557d1a-9643-456c-9b64-920456a621c7 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:55.114753 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 463/924] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:55.120401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ee95f-22e8-4b41-a4b2-76a83e045102 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:55.120678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ee95f-22e8-4b41-a4b2-76a83e045102 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:55.124125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ee95f-22e8-4b41-a4b2-76a83e045102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:55.132874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0557d1a-9643-456c-9b64-920456a621c7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:55.145225 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7f6ee95f-22e8-4b41-a4b2-76a83e045102 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:55.146567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7f6ee95f-22e8-4b41-a4b2-76a83e045102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:55.231629 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:55.261319 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 462/925] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:55.273556 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0557d1a-9643-456c-9b64-920456a621c7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:55.279742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ee95f-22e8-4b41-a4b2-76a83e045102 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:55.292757 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:55.317984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484297e9-0d91-4d85-a6d2-614113fb4992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:55.318941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484297e9-0d91-4d85-a6d2-614113fb4992 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:55.319328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484297e9-0d91-4d85-a6d2-614113fb4992 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:55.319552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484297e9-0d91-4d85-a6d2-614113fb4992 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:55.320491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484297e9-0d91-4d85-a6d2-614113fb4992 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:55.322184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484297e9-0d91-4d85-a6d2-614113fb4992 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:55.394582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 464/926] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:53:55 2022] POST /identity/v3/services => generated 241 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:53:55.398548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ee95f-22e8-4b41-a4b2-76a83e045102 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:55.448048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42d20b-211b-4309-ae9b-53e3290f2ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:55.448716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42d20b-211b-4309-ae9b-53e3290f2ebc None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:55.449029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42d20b-211b-4309-ae9b-53e3290f2ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:55.449320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42d20b-211b-4309-ae9b-53e3290f2ebc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:55.463554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42d20b-211b-4309-ae9b-53e3290f2ebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8e42d20b-211b-4309-ae9b-53e3290f2ebc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:55.468740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:55.469782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:55.470658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 463/927] 10.222.0.86 () {56 vars in 1120 bytes} [Thu Jul 28 17:53:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:55.477954 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484297e9-0d91-4d85-a6d2-614113fb4992 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:55.594141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484297e9-0d91-4d85-a6d2-614113fb4992 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:55.626991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:55.632825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb4486-e82d-4cad-a759-fd60c77b4d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:55.633549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb4486-e82d-4cad-a759-fd60c77b4d42 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:55.633842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb4486-e82d-4cad-a759-fd60c77b4d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:55.634123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb4486-e82d-4cad-a759-fd60c77b4d42 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:55.634855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb4486-e82d-4cad-a759-fd60c77b4d42 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:55.635168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb4486-e82d-4cad-a759-fd60c77b4d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:55.637793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb4486-e82d-4cad-a759-fd60c77b4d42 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:55.647545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 465/928] 10.222.0.86 () {56 vars in 1115 bytes} [Thu Jul 28 17:53:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:55.651722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42d20b-211b-4309-ae9b-53e3290f2ebc None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:55.719180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:55.759541 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42d20b-211b-4309-ae9b-53e3290f2ebc None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:55.796128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7b9e6-4080-4bfb-bab7-de178a139a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:55.796782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7b9e6-4080-4bfb-bab7-de178a139a81 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:55.797097 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7b9e6-4080-4bfb-bab7-de178a139a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:55.797376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7b9e6-4080-4bfb-bab7-de178a139a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:55.808191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7b9e6-4080-4bfb-bab7-de178a139a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:55.809688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7b9e6-4080-4bfb-bab7-de178a139a81 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-43e7b9e6-4080-4bfb-bab7-de178a139a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:55.825704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:55.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:55.827550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:55.827550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:55.827550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:55.827550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:55.827550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 464/929] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:55.834129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb4486-e82d-4cad-a759-fd60c77b4d42 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:55.941065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb4486-e82d-4cad-a759-fd60c77b4d42 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:55.977133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88307e0c-9810-4575-b356-19660c14c956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:55.978126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307e0c-9810-4575-b356-19660c14c956 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:55.978630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307e0c-9810-4575-b356-19660c14c956 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:55.979104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307e0c-9810-4575-b356-19660c14c956 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:55.980018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88307e0c-9810-4575-b356-19660c14c956 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:55.980524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88307e0c-9810-4575-b356-19660c14c956 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:55.982670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88307e0c-9810-4575-b356-19660c14c956 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:55.990037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e56f61-e77a-4ee3-9200-90cddb5ca320 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:55.990624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e56f61-e77a-4ee3-9200-90cddb5ca320 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:55.991116 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e56f61-e77a-4ee3-9200-90cddb5ca320 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:55.992931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 465/930] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:53:55.998184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a62e-a30a-4f04-9c00-c03d74b06614 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:55.998841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 466/931] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:55.999086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a62e-a30a-4f04-9c00-c03d74b06614 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:55.999811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a62e-a30a-4f04-9c00-c03d74b06614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.006239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88307e0c-9810-4575-b356-19660c14c956 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.019924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a7b8a62e-a30a-4f04-9c00-c03d74b06614 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:56.021145 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a7b8a62e-a30a-4f04-9c00-c03d74b06614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:56.067208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:56.097299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 466/932] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:56.114655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a62e-a30a-4f04-9c00-c03d74b06614 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.125300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88307e0c-9810-4575-b356-19660c14c956 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.154682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc665fc-3496-42d4-a910-8a90d435bea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.155659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc665fc-3496-42d4-a910-8a90d435bea0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.156144 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc665fc-3496-42d4-a910-8a90d435bea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.156646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc665fc-3496-42d4-a910-8a90d435bea0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.163775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:56.165645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc665fc-3496-42d4-a910-8a90d435bea0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.167575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc665fc-3496-42d4-a910-8a90d435bea0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-fbc665fc-3496-42d4-a910-8a90d435bea0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:56.180611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:56.181633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:56.182580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:56.182580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:56.182580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:56.182580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:56.182580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:56.182580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:56.182580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:56.182580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:56.182580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:53:56.182580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:56.185252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 467/933] 10.222.0.86 () {56 vars in 1117 bytes} [Thu Jul 28 17:53:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:56.190636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc665fc-3496-42d4-a910-8a90d435bea0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.231725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a62e-a30a-4f04-9c00-c03d74b06614 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.269723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5f122-cdee-429f-86da-e13761bd22c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.270380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5f122-cdee-429f-86da-e13761bd22c1 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.270643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5f122-cdee-429f-86da-e13761bd22c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.270892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5f122-cdee-429f-86da-e13761bd22c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.271590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5f122-cdee-429f-86da-e13761bd22c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.273021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5f122-cdee-429f-86da-e13761bd22c1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:56.275950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: WARNING keystone.server.flask.common [None req-c3d5f122-cdee-429f-86da-e13761bd22c1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 17:53:56.311219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc665fc-3496-42d4-a910-8a90d435bea0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.324566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:56.330351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.sql.core [None req-c3d5f122-cdee-429f-86da-e13761bd22c1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 17:53:56.330351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 17:53:56.330351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [parameters: {'id': '062d6e9ae5704a8481c8a909588da376', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 17:53:56.330351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80464) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c3d5f122-cdee-429f-86da-e13761bd22c1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 062d6e9ae5704a8481c8a909588da376.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 062d6e9ae5704a8481c8a909588da376. Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:53:56.338012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result.read() Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 17:53:56.339203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application yield resource Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise self.value Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application session.commit() Jul 28 17:53:56.340384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.session.flush() Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application compat.raise_( Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 17:53:56.342175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 17:53:56.343284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result.read() Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:53:56.344194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [parameters: {'id': '062d6e9ae5704a8481c8a909588da376', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:56.345159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:56.346041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:56.346954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:56.346954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 17:53:56.346954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 17:53:56.346954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:56.346954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:56.346954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 17:53:56.346954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 17:53:56.346954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 062d6e9ae5704a8481c8a909588da376. Jul 28 17:53:56.346954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:56.350049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 467/934] 10.222.0.86 () {60 vars in 1140 bytes} [Thu Jul 28 17:53:56 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:56.351310 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5f122-cdee-429f-86da-e13761bd22c1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.361147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f4356-c984-4a7a-89c9-5a0d8592148d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.361987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f4356-c984-4a7a-89c9-5a0d8592148d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.362456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f4356-c984-4a7a-89c9-5a0d8592148d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.362980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f4356-c984-4a7a-89c9-5a0d8592148d None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.364053 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f4356-c984-4a7a-89c9-5a0d8592148d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:56.364520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f4356-c984-4a7a-89c9-5a0d8592148d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.366143 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f4356-c984-4a7a-89c9-5a0d8592148d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:56.385039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 468/935] 10.222.0.86 () {56 vars in 1149 bytes} [Thu Jul 28 17:53:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:56.391449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f4356-c984-4a7a-89c9-5a0d8592148d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.476432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5f122-cdee-429f-86da-e13761bd22c1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.478165 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:56.507764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f4356-c984-4a7a-89c9-5a0d8592148d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.517225 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bdef4-fa35-4ff8-8434-1e02d8b8e0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.518134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bdef4-fa35-4ff8-8434-1e02d8b8e0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.518566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bdef4-fa35-4ff8-8434-1e02d8b8e0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.518989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bdef4-fa35-4ff8-8434-1e02d8b8e0b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.526496 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bdef4-fa35-4ff8-8434-1e02d8b8e0b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.528420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bdef4-fa35-4ff8-8434-1e02d8b8e0b2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-944bdef4-fa35-4ff8-8434-1e02d8b8e0b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:56.541787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:56.542796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:56.543923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:56.543923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:56.543923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:56.543923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:56.543923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:56.543923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:56.543923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:56.543923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:56.546319 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 468/936] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:56.554308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bdef4-fa35-4ff8-8434-1e02d8b8e0b2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.558363 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5540a-a205-410a-8ac3-30371edb769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.560044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5540a-a205-410a-8ac3-30371edb769b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.560360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5540a-a205-410a-8ac3-30371edb769b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.560675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5540a-a205-410a-8ac3-30371edb769b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.573720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5540a-a205-410a-8ac3-30371edb769b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.576223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5540a-a205-410a-8ac3-30371edb769b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-dab5540a-a205-410a-8ac3-30371edb769b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:53:56.596939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:56.603869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:56.604802 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:53:56.604802 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:53:56.604802 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:53:56.604802 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:56.604802 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 469/937] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:56.604802 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5540a-a205-410a-8ac3-30371edb769b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.643672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:56.671823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bdef4-fa35-4ff8-8434-1e02d8b8e0b2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.712573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec3ca4-2c53-4be6-bd51-a45c27eb0227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.713055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec3ca4-2c53-4be6-bd51-a45c27eb0227 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.713346 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec3ca4-2c53-4be6-bd51-a45c27eb0227 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.713628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec3ca4-2c53-4be6-bd51-a45c27eb0227 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.714359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec3ca4-2c53-4be6-bd51-a45c27eb0227 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:56.714671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec3ca4-2c53-4be6-bd51-a45c27eb0227 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.716153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec3ca4-2c53-4be6-bd51-a45c27eb0227 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:56.737368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 469/938] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:53:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:56.754184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5540a-a205-410a-8ac3-30371edb769b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.794828 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e5aaf-115b-4bee-b75f-c8b5c2d91804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.795554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e5aaf-115b-4bee-b75f-c8b5c2d91804 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.795951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e5aaf-115b-4bee-b75f-c8b5c2d91804 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.796343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e5aaf-115b-4bee-b75f-c8b5c2d91804 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.797223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e5aaf-115b-4bee-b75f-c8b5c2d91804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:56.797669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e5aaf-115b-4bee-b75f-c8b5c2d91804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.799195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e5aaf-115b-4bee-b75f-c8b5c2d91804 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:56.814944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 470/939] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:56.822124 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec3ca4-2c53-4be6-bd51-a45c27eb0227 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.850734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:56.938509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec3ca4-2c53-4be6-bd51-a45c27eb0227 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.976653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f27a64-7bd6-4c25-b81e-eac8828384ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.977492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27a64-7bd6-4c25-b81e-eac8828384ef None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.977890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27a64-7bd6-4c25-b81e-eac8828384ef None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.978295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27a64-7bd6-4c25-b81e-eac8828384ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.986017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f27a64-7bd6-4c25-b81e-eac8828384ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.987645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f27a64-7bd6-4c25-b81e-eac8828384ef None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-32f27a64-7bd6-4c25-b81e-eac8828384ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:53:56.999280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:57.000623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:57.001785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:53:57.001785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:53:57.001785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:53:57.001785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:53:57.001785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:53:57.001785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:53:57.001785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:53:57.001785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:57.003662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 470/940] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:57.010297 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e5aaf-115b-4bee-b75f-c8b5c2d91804 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.099717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:57.118275 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e5aaf-115b-4bee-b75f-c8b5c2d91804 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:57.155690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dead168-b2c6-40c3-8d32-95892cc2b700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:57.156502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dead168-b2c6-40c3-8d32-95892cc2b700 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.156887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dead168-b2c6-40c3-8d32-95892cc2b700 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.157307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dead168-b2c6-40c3-8d32-95892cc2b700 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.158199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dead168-b2c6-40c3-8d32-95892cc2b700 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:57.158623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dead168-b2c6-40c3-8d32-95892cc2b700 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:57.160180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dead168-b2c6-40c3-8d32-95892cc2b700 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:57.173325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 471/941] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:53:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:57.178909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f27a64-7bd6-4c25-b81e-eac8828384ef None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.261705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6404e-df7a-4b0d-9783-aa03c3498b31 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.262165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6404e-df7a-4b0d-9783-aa03c3498b31 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.262555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6404e-df7a-4b0d-9783-aa03c3498b31 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.264195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 472/942] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:57.268130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:57.270613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9b70e-70de-4c3e-ba89-680a3a1d708f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.271062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9b70e-70de-4c3e-ba89-680a3a1d708f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.271564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9b70e-70de-4c3e-ba89-680a3a1d708f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.289110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f27a64-7bd6-4c25-b81e-eac8828384ef None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:57.290972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-43e9b70e-70de-4c3e-ba89-680a3a1d708f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:57.293219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-43e9b70e-70de-4c3e-ba89-680a3a1d708f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:57.329774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53076b-940a-4d5b-a67b-61717aa5d8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:57.330722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53076b-940a-4d5b-a67b-61717aa5d8bd None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.331172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53076b-940a-4d5b-a67b-61717aa5d8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.331581 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53076b-940a-4d5b-a67b-61717aa5d8bd None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/e3311c1ac5e349fa9ba83a5820171b61/roles/8e5389a3e81c4e58b716a69b7f74257b` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.334151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53076b-940a-4d5b-a67b-61717aa5d8bd None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=e3311c1ac5e349fa9ba83a5820171b61, role_id=8e5389a3e81c4e58b716a69b7f74257b)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:57.335751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53076b-940a-4d5b-a67b-61717aa5d8bd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:57.369081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 473/943] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:53:57.389863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9b70e-70de-4c3e-ba89-680a3a1d708f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.413054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 471/944] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:53:57 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/e3311c1ac5e349fa9ba83a5820171b61/roles/8e5389a3e81c4e58b716a69b7f74257b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:57.483631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:57.517754 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9b70e-70de-4c3e-ba89-680a3a1d708f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:57.559025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51a61c-798a-4ea7-907f-31104e6cb801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:57.560444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51a61c-798a-4ea7-907f-31104e6cb801 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.561262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51a61c-798a-4ea7-907f-31104e6cb801 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.562054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51a61c-798a-4ea7-907f-31104e6cb801 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.563537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51a61c-798a-4ea7-907f-31104e6cb801 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:57.565837 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51a61c-798a-4ea7-907f-31104e6cb801 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ea51a61c-798a-4ea7-907f-31104e6cb801 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:57.576601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:57.577714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:53:57.578570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:57.582538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 474/945] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:53:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:57.588775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53076b-940a-4d5b-a67b-61717aa5d8bd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.608509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87b5fe-6e02-4f18-8caa-0f54b70acdf2 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.608840 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87b5fe-6e02-4f18-8caa-0f54b70acdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.609130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87b5fe-6e02-4f18-8caa-0f54b70acdf2 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.610616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 475/946] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:57.615904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732345ba-5b98-47c1-8fbf-d146b4746b1c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.616185 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732345ba-5b98-47c1-8fbf-d146b4746b1c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.616458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732345ba-5b98-47c1-8fbf-d146b4746b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.635322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-732345ba-5b98-47c1-8fbf-d146b4746b1c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:57.636452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-732345ba-5b98-47c1-8fbf-d146b4746b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:57.710746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 476/947] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:53:57.720197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53076b-940a-4d5b-a67b-61717aa5d8bd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:57.739330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732345ba-5b98-47c1-8fbf-d146b4746b1c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.758611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb572d-ebaf-4954-bd30-936a097283ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:57.759248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb572d-ebaf-4954-bd30-936a097283ed None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.759535 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb572d-ebaf-4954-bd30-936a097283ed None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.759813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb572d-ebaf-4954-bd30-936a097283ed None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.760505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb572d-ebaf-4954-bd30-936a097283ed None admin] RBAC: Adding query filter params (name=compute) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:57.760808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb572d-ebaf-4954-bd30-936a097283ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:57.762227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb572d-ebaf-4954-bd30-936a097283ed None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:57.767865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 472/948] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:53:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:57.773711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb572d-ebaf-4954-bd30-936a097283ed None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.783827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:57.830718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:57.868760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732345ba-5b98-47c1-8fbf-d146b4746b1c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:57.900157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb572d-ebaf-4954-bd30-936a097283ed None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:57.912530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dbf87-743d-4758-9fe1-45cad347559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:57.913310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dbf87-743d-4758-9fe1-45cad347559c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.913733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dbf87-743d-4758-9fe1-45cad347559c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.914292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dbf87-743d-4758-9fe1-45cad347559c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.914858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dbf87-743d-4758-9fe1-45cad347559c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:57.916718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dbf87-743d-4758-9fe1-45cad347559c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a16dbf87-743d-4758-9fe1-45cad347559c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:57.929856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:57.930921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:57.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 477/949] 10.222.0.86 () {56 vars in 1126 bytes} [Thu Jul 28 17:53:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:53:57.938981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dbf87-743d-4758-9fe1-45cad347559c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.956308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b60f-7838-434e-8855-870cb17863cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:57.957500 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b60f-7838-434e-8855-870cb17863cd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.957971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b60f-7838-434e-8855-870cb17863cd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.958838 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b60f-7838-434e-8855-870cb17863cd None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.960140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8b60f-7838-434e-8855-870cb17863cd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:57.960646 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8b60f-7838-434e-8855-870cb17863cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:57.962251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8b60f-7838-434e-8855-870cb17863cd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:57.969773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 473/950] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:53:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:58.065724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dbf87-743d-4758-9fe1-45cad347559c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.098982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:58.105280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79f74c-7a20-4576-b20d-4f4fb104ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.105998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79f74c-7a20-4576-b20d-4f4fb104ef80 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.106390 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79f74c-7a20-4576-b20d-4f4fb104ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.106705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79f74c-7a20-4576-b20d-4f4fb104ef80 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.107468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79f74c-7a20-4576-b20d-4f4fb104ef80 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:58.107773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79f74c-7a20-4576-b20d-4f4fb104ef80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.109269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79f74c-7a20-4576-b20d-4f4fb104ef80 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.115190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 478/951] 10.222.0.86 () {56 vars in 1122 bytes} [Thu Jul 28 17:53:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:58.122516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b60f-7838-434e-8855-870cb17863cd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.207268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:58.243612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b60f-7838-434e-8855-870cb17863cd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.280414 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119727a-0f60-4265-9e5e-acc32d20e131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.281067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119727a-0f60-4265-9e5e-acc32d20e131 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.281351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119727a-0f60-4265-9e5e-acc32d20e131 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.281629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119727a-0f60-4265-9e5e-acc32d20e131 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.282374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119727a-0f60-4265-9e5e-acc32d20e131 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:58.282737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119727a-0f60-4265-9e5e-acc32d20e131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.284170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119727a-0f60-4265-9e5e-acc32d20e131 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.289459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 474/952] 10.222.0.86 () {56 vars in 1122 bytes} [Thu Jul 28 17:53:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:58.295738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79f74c-7a20-4576-b20d-4f4fb104ef80 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.403503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79f74c-7a20-4576-b20d-4f4fb104ef80 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.437408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7a43c-bc98-413e-a977-45ccdb4c12e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.438028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7a43c-bc98-413e-a977-45ccdb4c12e4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.438309 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7a43c-bc98-413e-a977-45ccdb4c12e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.438579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7a43c-bc98-413e-a977-45ccdb4c12e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.439335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7a43c-bc98-413e-a977-45ccdb4c12e4 None admin] RBAC: Adding query filter params (interface=public, service_id=3549fa2830cf41c1ac64c6ee0571f5e5, region_id=RegionOne) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:58.439641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7a43c-bc98-413e-a977-45ccdb4c12e4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.441016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7a43c-bc98-413e-a977-45ccdb4c12e4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.446541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 479/953] 10.222.0.86 () {56 vars in 1251 bytes} [Thu Jul 28 17:53:58 2022] GET /identity/v3/endpoints?service_id=3549fa2830cf41c1ac64c6ee0571f5e5&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:58.452567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119727a-0f60-4265-9e5e-acc32d20e131 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.561994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119727a-0f60-4265-9e5e-acc32d20e131 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.600056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44787e1a-4c82-425f-bbfa-604c6d38cc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.600762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44787e1a-4c82-425f-bbfa-604c6d38cc76 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.601094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44787e1a-4c82-425f-bbfa-604c6d38cc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.601411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44787e1a-4c82-425f-bbfa-604c6d38cc76 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.602070 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44787e1a-4c82-425f-bbfa-604c6d38cc76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.603496 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44787e1a-4c82-425f-bbfa-604c6d38cc76 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.610675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 475/954] 10.222.0.86 () {56 vars in 1087 bytes} [Thu Jul 28 17:53:58 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:58.611048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:58.966294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c75b4e-7291-4ac1-81de-6e511db9edc2 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.966823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c75b4e-7291-4ac1-81de-6e511db9edc2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.966903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c75b4e-7291-4ac1-81de-6e511db9edc2 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.968584 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 480/955] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:58.974183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbeb14-a308-491b-b3ad-0db5f4043446 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.974470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbeb14-a308-491b-b3ad-0db5f4043446 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.974743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbeb14-a308-491b-b3ad-0db5f4043446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.992897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f0cbeb14-a308-491b-b3ad-0db5f4043446 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:58.994448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f0cbeb14-a308-491b-b3ad-0db5f4043446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:59.063976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 476/956] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:53:59.088875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c75b4e-7291-4ac1-81de-6e511db9edc2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:59.152572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:59.197577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c75b4e-7291-4ac1-81de-6e511db9edc2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:59.234146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414421df-f360-40f6-9f76-82752b87935c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:59.234690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414421df-f360-40f6-9f76-82752b87935c None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:59.234971 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414421df-f360-40f6-9f76-82752b87935c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:59.235248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414421df-f360-40f6-9f76-82752b87935c None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:59.235949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414421df-f360-40f6-9f76-82752b87935c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:59.237416 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414421df-f360-40f6-9f76-82752b87935c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:59.239976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: WARNING keystone.server.flask.common [None req-414421df-f360-40f6-9f76-82752b87935c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 17:53:59.284295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.sql.core [None req-414421df-f360-40f6-9f76-82752b87935c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 17:53:59.284295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 17:53:59.284295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [parameters: {'user_id': '7b1fccc2aa1f40ac850e02032e880775', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 17:53:59.284295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80463) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-414421df-f360-40f6-9f76-82752b87935c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result.read() Jul 28 17:53:59.302415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:53:59.303387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application yield resource Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise self.value Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application session.commit() Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 17:53:59.304490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.session.flush() Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application compat.raise_( Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 17:53:59.305437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:53:59.306364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result.read() Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:53:59.307236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [parameters: {'user_id': '7b1fccc2aa1f40ac850e02032e880775', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:59.308389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 17:53:59.310834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 17:53:59.313166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 17:53:59.313166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 17:53:59.313166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 17:53:59.313166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 17:53:59.313166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 17:53:59.313166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:53:59.313166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 481/957] 10.222.0.86 () {60 vars in 1131 bytes} [Thu Jul 28 17:53:59 2022] POST /identity/v3/users => generated 160 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:59.314062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cbeb14-a308-491b-b3ad-0db5f4043446 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:59.424720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cbeb14-a308-491b-b3ad-0db5f4043446 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:59.454307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:59.462144 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284912f-0444-440d-ba47-f330033a9d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:59.462803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284912f-0444-440d-ba47-f330033a9d94 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:59.463090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284912f-0444-440d-ba47-f330033a9d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:59.463371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284912f-0444-440d-ba47-f330033a9d94 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:59.470999 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284912f-0444-440d-ba47-f330033a9d94 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:59.472446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284912f-0444-440d-ba47-f330033a9d94 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a284912f-0444-440d-ba47-f330033a9d94 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:53:59.485006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:59.485953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:59.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:59.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:59.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:53:59.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:53:59.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:53:59.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:53:59.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:53:59.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:53:59.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:53:59.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:59.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 477/958] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:53:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:59.492455 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414421df-f360-40f6-9f76-82752b87935c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:59.531472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109577f-a772-4eff-8a60-a48a35693275 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:59.531847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109577f-a772-4eff-8a60-a48a35693275 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:59.532065 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109577f-a772-4eff-8a60-a48a35693275 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:59.533678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 478/959] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:53:59.540322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec7e1e-b9d9-4527-b614-0040b6b1468c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:59.540619 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec7e1e-b9d9-4527-b614-0040b6b1468c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:59.540874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec7e1e-b9d9-4527-b614-0040b6b1468c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:59.569700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-64ec7e1e-b9d9-4527-b614-0040b6b1468c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:59.572475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-64ec7e1e-b9d9-4527-b614-0040b6b1468c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:59.602361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414421df-f360-40f6-9f76-82752b87935c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:59.624860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:59.639033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03018221-c963-4c3a-8de7-635ad52baf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:59.639664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018221-c963-4c3a-8de7-635ad52baf29 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:59.639948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018221-c963-4c3a-8de7-635ad52baf29 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:59.640224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018221-c963-4c3a-8de7-635ad52baf29 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:59.641021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03018221-c963-4c3a-8de7-635ad52baf29 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:59.641333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03018221-c963-4c3a-8de7-635ad52baf29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:59.643035 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03018221-c963-4c3a-8de7-635ad52baf29 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:59.670562 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 479/960] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:53:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:53:59.684802 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 482/961] 10.222.0.86 () {56 vars in 1101 bytes} [Thu Jul 28 17:53:59 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:59.695540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec7e1e-b9d9-4527-b614-0040b6b1468c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:59.816734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec7e1e-b9d9-4527-b614-0040b6b1468c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:59.840932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:59.858916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee92f3-fecb-4e92-a382-bbd779f763ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:59.859594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee92f3-fecb-4e92-a382-bbd779f763ec None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:59.860588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee92f3-fecb-4e92-a382-bbd779f763ec None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:59.860588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee92f3-fecb-4e92-a382-bbd779f763ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:59.870890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee92f3-fecb-4e92-a382-bbd779f763ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-55ee92f3-fecb-4e92-a382-bbd779f763ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:53:59.876716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:53:59.877695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:53:59.878702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 480/962] 10.222.0.86 () {56 vars in 1120 bytes} [Thu Jul 28 17:53:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:53:59.884145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03018221-c963-4c3a-8de7-635ad52baf29 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:59.990325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03018221-c963-4c3a-8de7-635ad52baf29 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:00.025675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96454fcf-acc1-431c-9998-7811b9011f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:00.026294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96454fcf-acc1-431c-9998-7811b9011f49 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.026577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96454fcf-acc1-431c-9998-7811b9011f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.026856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96454fcf-acc1-431c-9998-7811b9011f49 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.027584 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96454fcf-acc1-431c-9998-7811b9011f49 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:00.027896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96454fcf-acc1-431c-9998-7811b9011f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:00.029335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96454fcf-acc1-431c-9998-7811b9011f49 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:00.042281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 483/963] 10.222.0.86 () {56 vars in 1115 bytes} [Thu Jul 28 17:53:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:00.043436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee92f3-fecb-4e92-a382-bbd779f763ec None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:00.147590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee92f3-fecb-4e92-a382-bbd779f763ec None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:00.178470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6f0b-edd4-4955-8b2d-b36a81f087b7 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.178807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6f0b-edd4-4955-8b2d-b36a81f087b7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.179060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6f0b-edd4-4955-8b2d-b36a81f087b7 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.180682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 484/964] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:54:00.184503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ef8f-2312-41b3-a5ba-cedabf44d665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:00.185745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ef8f-2312-41b3-a5ba-cedabf44d665 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.186029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ef8f-2312-41b3-a5ba-cedabf44d665 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.186311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ef8f-2312-41b3-a5ba-cedabf44d665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.186413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2289d-1503-4d54-95ed-e8cadbc885e4 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.186664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2289d-1503-4d54-95ed-e8cadbc885e4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.187049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2289d-1503-4d54-95ed-e8cadbc885e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.198644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4ef8f-2312-41b3-a5ba-cedabf44d665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:00.200235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4ef8f-2312-41b3-a5ba-cedabf44d665 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:00.209880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4fb2289d-1503-4d54-95ed-e8cadbc885e4 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:00.211572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4fb2289d-1503-4d54-95ed-e8cadbc885e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-7fd4ef8f-2312-41b3-a5ba-cedabf44d665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:54:00.215328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:00.216360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:00.221891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:54:00.221891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:54:00.221891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:00.221891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:00.221891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 481/965] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:00.222175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ef8f-2312-41b3-a5ba-cedabf44d665 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:00.253528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:00.255808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:00.286878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 485/966] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:00.309413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2289d-1503-4d54-95ed-e8cadbc885e4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:00.333240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ef8f-2312-41b3-a5ba-cedabf44d665 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:00.369435 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd42f5-9d5a-43b5-8c3a-65f2ddc777ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:00.370155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd42f5-9d5a-43b5-8c3a-65f2ddc777ba None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.370448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd42f5-9d5a-43b5-8c3a-65f2ddc777ba None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.370730 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd42f5-9d5a-43b5-8c3a-65f2ddc777ba None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.371434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd42f5-9d5a-43b5-8c3a-65f2ddc777ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:00.371740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd42f5-9d5a-43b5-8c3a-65f2ddc777ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:00.373722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd42f5-9d5a-43b5-8c3a-65f2ddc777ba None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:00.387058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 482/967] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:00.392513 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd42f5-9d5a-43b5-8c3a-65f2ddc777ba None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:00.427941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2289d-1503-4d54-95ed-e8cadbc885e4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:00.464615 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bcd4e-10b4-4e95-8798-fe7ab84c20c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:00.465290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bcd4e-10b4-4e95-8798-fe7ab84c20c1 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.465561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bcd4e-10b4-4e95-8798-fe7ab84c20c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.466352 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bcd4e-10b4-4e95-8798-fe7ab84c20c1 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.467091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557bcd4e-10b4-4e95-8798-fe7ab84c20c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:00.468618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557bcd4e-10b4-4e95-8798-fe7ab84c20c1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:00.488467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:00.497630 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd42f5-9d5a-43b5-8c3a-65f2ddc777ba None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:00.531669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b67f7-c4b7-4cae-86b6-0dcb69e9e1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:00.532338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b67f7-c4b7-4cae-86b6-0dcb69e9e1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.532622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b67f7-c4b7-4cae-86b6-0dcb69e9e1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.532898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b67f7-c4b7-4cae-86b6-0dcb69e9e1b0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.539901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b67f7-c4b7-4cae-86b6-0dcb69e9e1b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:00.541378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b67f7-c4b7-4cae-86b6-0dcb69e9e1b0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-4b9b67f7-c4b7-4cae-86b6-0dcb69e9e1b0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:54:00.554101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:00.555201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:00.561641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:00.561641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:00.561641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:54:00.561641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:54:00.561641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:54:00.561641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:54:00.561641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:54:00.561641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:54:00.561641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:54:00.561641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:00.561641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 483/968] 10.222.0.86 () {56 vars in 1117 bytes} [Thu Jul 28 17:54:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:00.562346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 486/969] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:54:00 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:54:00.562425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b67f7-c4b7-4cae-86b6-0dcb69e9e1b0 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:00.677192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b67f7-c4b7-4cae-86b6-0dcb69e9e1b0 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:00.721906 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d01ea-7dd2-4f14-9a97-e914edf7d287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:00.722536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d01ea-7dd2-4f14-9a97-e914edf7d287 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.722832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d01ea-7dd2-4f14-9a97-e914edf7d287 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.723096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d01ea-7dd2-4f14-9a97-e914edf7d287 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.723974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d01ea-7dd2-4f14-9a97-e914edf7d287 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:00.724292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d01ea-7dd2-4f14-9a97-e914edf7d287 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:00.725203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b6d12-c4b4-4aa8-a377-a1b766208268 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.725494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b6d12-c4b4-4aa8-a377-a1b766208268 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.725752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d01ea-7dd2-4f14-9a97-e914edf7d287 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:00.725857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b6d12-c4b4-4aa8-a377-a1b766208268 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.727314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 487/970] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:00.733091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91106fc-19e0-4425-b8d7-58abe9774f8d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.733378 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91106fc-19e0-4425-b8d7-58abe9774f8d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.733647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91106fc-19e0-4425-b8d7-58abe9774f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.745094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 484/971] 10.222.0.86 () {56 vars in 1149 bytes} [Thu Jul 28 17:54:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:00.750910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d01ea-7dd2-4f14-9a97-e914edf7d287 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:00.754515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:00.757752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f91106fc-19e0-4425-b8d7-58abe9774f8d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:00.757752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f91106fc-19e0-4425-b8d7-58abe9774f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:00.782560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:00.829093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 488/972] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:00.852559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91106fc-19e0-4425-b8d7-58abe9774f8d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:00.865715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d01ea-7dd2-4f14-9a97-e914edf7d287 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:00.903205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b8011-3ed0-4fc5-aee6-681991a8688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:00.904401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b8011-3ed0-4fc5-aee6-681991a8688f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.904680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b8011-3ed0-4fc5-aee6-681991a8688f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.904955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b8011-3ed0-4fc5-aee6-681991a8688f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.958644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91106fc-19e0-4425-b8d7-58abe9774f8d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:00.978108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b8011-3ed0-4fc5-aee6-681991a8688f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:00.979544 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b8011-3ed0-4fc5-aee6-681991a8688f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:00.982315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:00.994463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520fb3-efed-436a-b763-84cda67082ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:00.995110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520fb3-efed-436a-b763-84cda67082ec None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:00.996234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520fb3-efed-436a-b763-84cda67082ec None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:00.996234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520fb3-efed-436a-b763-84cda67082ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:00.996418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00520fb3-efed-436a-b763-84cda67082ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:00.997795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00520fb3-efed-436a-b763-84cda67082ec None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-00520fb3-efed-436a-b763-84cda67082ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:01.006993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:01.009329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:01.010273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 489/973] 10.222.0.86 () {56 vars in 1126 bytes} [Thu Jul 28 17:54:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:01.014674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520fb3-efed-436a-b763-84cda67082ec None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:01.042502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:01.091698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 485/974] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:54:00 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:54:01.096812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b8011-3ed0-4fc5-aee6-681991a8688f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:01.124171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520fb3-efed-436a-b763-84cda67082ec None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:01.161316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6033b0-9ac4-40a8-b9de-f2cb93ac76f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:01.161950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6033b0-9ac4-40a8-b9de-f2cb93ac76f4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:01.162235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6033b0-9ac4-40a8-b9de-f2cb93ac76f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:01.162512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6033b0-9ac4-40a8-b9de-f2cb93ac76f4 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:01.163197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6033b0-9ac4-40a8-b9de-f2cb93ac76f4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:01.163500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6033b0-9ac4-40a8-b9de-f2cb93ac76f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:01.164862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6033b0-9ac4-40a8-b9de-f2cb93ac76f4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:01.169998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 490/975] 10.222.0.86 () {56 vars in 1122 bytes} [Thu Jul 28 17:54:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:01.175582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6033b0-9ac4-40a8-b9de-f2cb93ac76f4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:01.210197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b8011-3ed0-4fc5-aee6-681991a8688f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:01.213415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:01.248362 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555cd98-1554-477c-b59d-85d98f220bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:01.249148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555cd98-1554-477c-b59d-85d98f220bac None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:01.249697 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555cd98-1554-477c-b59d-85d98f220bac None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:01.249971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555cd98-1554-477c-b59d-85d98f220bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:01.260603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555cd98-1554-477c-b59d-85d98f220bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:01.262250 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555cd98-1554-477c-b59d-85d98f220bac None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:01.267255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:01.280176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6033b0-9ac4-40a8-b9de-f2cb93ac76f4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-5555cd98-1554-477c-b59d-85d98f220bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:54:01.280379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:01.281254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:01.282180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:54:01.282180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:54:01.282180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:01.282180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:01.286249 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 486/976] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:01.287093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555cd98-1554-477c-b59d-85d98f220bac None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:01.320716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7ce3d-a3e7-44f7-9e3d-4cb3ffd28387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:01.321384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7ce3d-a3e7-44f7-9e3d-4cb3ffd28387 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:01.321667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7ce3d-a3e7-44f7-9e3d-4cb3ffd28387 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:01.321941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7ce3d-a3e7-44f7-9e3d-4cb3ffd28387 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:01.322632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7ce3d-a3e7-44f7-9e3d-4cb3ffd28387 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:01.322936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7ce3d-a3e7-44f7-9e3d-4cb3ffd28387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:01.324345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7ce3d-a3e7-44f7-9e3d-4cb3ffd28387 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:01.329194 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 491/977] 10.222.0.86 () {56 vars in 1122 bytes} [Thu Jul 28 17:54:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:01.334711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7ce3d-a3e7-44f7-9e3d-4cb3ffd28387 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:01.396987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555cd98-1554-477c-b59d-85d98f220bac None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:01.438270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b165a54-963c-4258-942f-ac0177c482fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:01.438905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b165a54-963c-4258-942f-ac0177c482fb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:01.439189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b165a54-963c-4258-942f-ac0177c482fb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:01.439471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b165a54-963c-4258-942f-ac0177c482fb None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:01.440166 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b165a54-963c-4258-942f-ac0177c482fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:01.440477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b165a54-963c-4258-942f-ac0177c482fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:01.441887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b165a54-963c-4258-942f-ac0177c482fb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:01.444652 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7ce3d-a3e7-44f7-9e3d-4cb3ffd28387 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:01.453854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 487/978] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:01.460089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b165a54-963c-4258-942f-ac0177c482fb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:01.479298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:01.483369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cdb80-95ee-472c-a4be-0e2b28a7aae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:01.484364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cdb80-95ee-472c-a4be-0e2b28a7aae9 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:01.484364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cdb80-95ee-472c-a4be-0e2b28a7aae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:01.484647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cdb80-95ee-472c-a4be-0e2b28a7aae9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:01.485267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9cdb80-95ee-472c-a4be-0e2b28a7aae9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:01.486641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9cdb80-95ee-472c-a4be-0e2b28a7aae9 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:01.562653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b165a54-963c-4258-942f-ac0177c482fb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:01.574441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 492/979] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 17:54:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:54:01.626306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d101d-dfef-41ce-a6c7-c7f617ead57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:01.626976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d101d-dfef-41ce-a6c7-c7f617ead57b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:01.627259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d101d-dfef-41ce-a6c7-c7f617ead57b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:01.627534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d101d-dfef-41ce-a6c7-c7f617ead57b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:01.635789 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d101d-dfef-41ce-a6c7-c7f617ead57b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:01.637297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d101d-dfef-41ce-a6c7-c7f617ead57b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-fd1d101d-dfef-41ce-a6c7-c7f617ead57b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:54:01.651205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:01.652374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:01.654354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:54:01.654354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:54:01.654354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:54:01.654354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:54:01.654354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:54:01.654354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:54:01.654354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:54:01.654354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:01.657389 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 488/980] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:54:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:01.658964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cdb80-95ee-472c-a4be-0e2b28a7aae9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:01.671694 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:01.769334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cdb80-95ee-472c-a4be-0e2b28a7aae9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:01.805124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26eef4-4420-4f15-9abc-14395c313bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:01.805763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26eef4-4420-4f15-9abc-14395c313bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:01.806053 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26eef4-4420-4f15-9abc-14395c313bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:01.806331 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26eef4-4420-4f15-9abc-14395c313bf0 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:01.807112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26eef4-4420-4f15-9abc-14395c313bf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:01.807424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26eef4-4420-4f15-9abc-14395c313bf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:01.808859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26eef4-4420-4f15-9abc-14395c313bf0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:01.821460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 493/981] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:54:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:01.828004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d101d-dfef-41ce-a6c7-c7f617ead57b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:01.904732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:01.937997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d101d-dfef-41ce-a6c7-c7f617ead57b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:01.979257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbde519-32c7-43fe-ab90-65e30072406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:01.979983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbde519-32c7-43fe-ab90-65e30072406f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:01.980340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbde519-32c7-43fe-ab90-65e30072406f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:01.980690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbde519-32c7-43fe-ab90-65e30072406f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:01.982722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbde519-32c7-43fe-ab90-65e30072406f None admin] RBAC: Adding query filter params (role.id=8e5389a3e81c4e58b716a69b7f74257b, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=e3311c1ac5e349fa9ba83a5820171b61) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:01.983107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbde519-32c7-43fe-ab90-65e30072406f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:01.984613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbde519-32c7-43fe-ab90-65e30072406f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:01.993710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 489/982] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:54:01 2022] GET /identity/v3/role_assignments?user.id=e3311c1ac5e349fa9ba83a5820171b61&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=8e5389a3e81c4e58b716a69b7f74257b => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:02.613159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8699b-f893-4cb4-b82d-b4a434dd7aec None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:02.613708 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8699b-f893-4cb4-b82d-b4a434dd7aec None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:02.613708 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8699b-f893-4cb4-b82d-b4a434dd7aec None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:02.615223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 494/983] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:02.620254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333730-3988-4b19-8d70-35eee0096f2c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:02.620638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333730-3988-4b19-8d70-35eee0096f2c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:02.620816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333730-3988-4b19-8d70-35eee0096f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:02.639634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-85333730-3988-4b19-8d70-35eee0096f2c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:02.640813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-85333730-3988-4b19-8d70-35eee0096f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:02.709811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 490/984] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:02.733032 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8699b-f893-4cb4-b82d-b4a434dd7aec None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:02.801379 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:02.834830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8699b-f893-4cb4-b82d-b4a434dd7aec None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:02.869786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fe3ee-f21b-4f48-95c7-b803fd41d1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:02.870420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe3ee-f21b-4f48-95c7-b803fd41d1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:02.870700 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe3ee-f21b-4f48-95c7-b803fd41d1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:02.870971 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fe3ee-f21b-4f48-95c7-b803fd41d1a4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:02.871590 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fe3ee-f21b-4f48-95c7-b803fd41d1a4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:02.875243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fe3ee-f21b-4f48-95c7-b803fd41d1a4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-3a0fe3ee-f21b-4f48-95c7-b803fd41d1a4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:02.885302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:02.886323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:54:02.887337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:02.889228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 495/985] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:54:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:02.892460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85333730-3988-4b19-8d70-35eee0096f2c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:02.957314 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:02.997739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85333730-3988-4b19-8d70-35eee0096f2c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:03.033328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fe9d7-69e7-4073-a2b6-d37dfdc7b475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:03.034181 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fe9d7-69e7-4073-a2b6-d37dfdc7b475 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:03.034617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fe9d7-69e7-4073-a2b6-d37dfdc7b475 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:03.035048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fe9d7-69e7-4073-a2b6-d37dfdc7b475 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:03.035879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fe9d7-69e7-4073-a2b6-d37dfdc7b475 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:03.036349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fe9d7-69e7-4073-a2b6-d37dfdc7b475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:03.037913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fe9d7-69e7-4073-a2b6-d37dfdc7b475 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:03.043458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 491/986] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:54:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:03.049493 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fe3ee-f21b-4f48-95c7-b803fd41d1a4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:03.157213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fe3ee-f21b-4f48-95c7-b803fd41d1a4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:03.186331 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:03.192012 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4941eb-5635-4c4f-b38e-d97d5bd35ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:03.192641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4941eb-5635-4c4f-b38e-d97d5bd35ebd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:03.192927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4941eb-5635-4c4f-b38e-d97d5bd35ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:03.193233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4941eb-5635-4c4f-b38e-d97d5bd35ebd None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:03.193922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4941eb-5635-4c4f-b38e-d97d5bd35ebd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:03.194232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4941eb-5635-4c4f-b38e-d97d5bd35ebd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:03.195664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4941eb-5635-4c4f-b38e-d97d5bd35ebd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:03.201249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 496/987] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:54:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:03.206876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fe9d7-69e7-4073-a2b6-d37dfdc7b475 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:03.309829 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fe9d7-69e7-4073-a2b6-d37dfdc7b475 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:03.342560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:03.355530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d83c8-6b44-41a5-bf8a-f9594573cbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:03.356161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d83c8-6b44-41a5-bf8a-f9594573cbad None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:03.357102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d83c8-6b44-41a5-bf8a-f9594573cbad None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:03.357306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d83c8-6b44-41a5-bf8a-f9594573cbad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:03.358039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d83c8-6b44-41a5-bf8a-f9594573cbad None admin] RBAC: Adding query filter params (interface=public, service_id=ea0fc3f6475f495fbb9192c9f3cfd434, region_id=RegionOne) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:03.358351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d83c8-6b44-41a5-bf8a-f9594573cbad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:03.359716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d83c8-6b44-41a5-bf8a-f9594573cbad None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:03.369373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 492/988] 10.222.0.86 () {56 vars in 1251 bytes} [Thu Jul 28 17:54:03 2022] GET /identity/v3/endpoints?service_id=ea0fc3f6475f495fbb9192c9f3cfd434&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:03.370529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4941eb-5635-4c4f-b38e-d97d5bd35ebd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:03.478492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4941eb-5635-4c4f-b38e-d97d5bd35ebd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:03.513042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581c436-5f19-4b49-9ec7-596a9ac25e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:03.513669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581c436-5f19-4b49-9ec7-596a9ac25e80 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:03.513956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581c436-5f19-4b49-9ec7-596a9ac25e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:03.514238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581c436-5f19-4b49-9ec7-596a9ac25e80 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:03.514897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581c436-5f19-4b49-9ec7-596a9ac25e80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:03.516273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581c436-5f19-4b49-9ec7-596a9ac25e80 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:03.529041 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 497/989] 10.222.0.86 () {56 vars in 1087 bytes} [Thu Jul 28 17:54:03 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:54:03.635884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00febdf9-b397-48a5-ad0d-a6c72dfed48f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:03.636475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00febdf9-b397-48a5-ad0d-a6c72dfed48f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:03.636475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00febdf9-b397-48a5-ad0d-a6c72dfed48f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:03.638099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 493/990] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:03.643714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1535924-5224-45a2-81b3-80bc0dda6c5a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:03.644190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1535924-5224-45a2-81b3-80bc0dda6c5a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:03.644266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1535924-5224-45a2-81b3-80bc0dda6c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:03.663701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e1535924-5224-45a2-81b3-80bc0dda6c5a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:03.668153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e1535924-5224-45a2-81b3-80bc0dda6c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:03.726287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:03.738278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 498/991] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:03.760762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00febdf9-b397-48a5-ad0d-a6c72dfed48f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:03.865733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00febdf9-b397-48a5-ad0d-a6c72dfed48f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:03.900665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d08e4-2880-4f00-a40b-5f966ae7dd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:03.901259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d08e4-2880-4f00-a40b-5f966ae7dd06 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:03.901544 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d08e4-2880-4f00-a40b-5f966ae7dd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:03.901821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d08e4-2880-4f00-a40b-5f966ae7dd06 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:03.902456 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d08e4-2880-4f00-a40b-5f966ae7dd06 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:03.904448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d08e4-2880-4f00-a40b-5f966ae7dd06 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-7b0d08e4-2880-4f00-a40b-5f966ae7dd06 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:03.913271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:03.914370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:54:03.915274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:03.917043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 494/992] 10.222.0.86 () {56 vars in 1126 bytes} [Thu Jul 28 17:54:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:03.920371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1535924-5224-45a2-81b3-80bc0dda6c5a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:04.025750 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1535924-5224-45a2-81b3-80bc0dda6c5a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:04.066468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a8210-8cb8-4956-9bfa-db2e2e6b5033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:04.071415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a8210-8cb8-4956-9bfa-db2e2e6b5033 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:04.071415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a8210-8cb8-4956-9bfa-db2e2e6b5033 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:04.071415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a8210-8cb8-4956-9bfa-db2e2e6b5033 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:04.071415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a8210-8cb8-4956-9bfa-db2e2e6b5033 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:04.071415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a8210-8cb8-4956-9bfa-db2e2e6b5033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:04.071415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a8210-8cb8-4956-9bfa-db2e2e6b5033 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:04.081200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 499/993] 10.222.0.86 () {56 vars in 1122 bytes} [Thu Jul 28 17:54:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:04.082429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d08e4-2880-4f00-a40b-5f966ae7dd06 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:04.117165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee22934-bea2-447c-aedc-f7f5919c486d None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:04.117483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee22934-bea2-447c-aedc-f7f5919c486d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:04.117736 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee22934-bea2-447c-aedc-f7f5919c486d None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:04.119243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 500/994] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:54:04.124020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640a312-b1ad-4596-9a39-a1357d26bc1c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:04.124563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640a312-b1ad-4596-9a39-a1357d26bc1c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:04.124851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640a312-b1ad-4596-9a39-a1357d26bc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:04.143574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2640a312-b1ad-4596-9a39-a1357d26bc1c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:04.146344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2640a312-b1ad-4596-9a39-a1357d26bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:04.147089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:04.191424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d08e4-2880-4f00-a40b-5f966ae7dd06 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:04.199499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:04.229256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 501/995] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:04 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:54:04.230158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185134a9-ac49-41f8-b34c-afe527684466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:04.230913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185134a9-ac49-41f8-b34c-afe527684466 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:04.231136 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185134a9-ac49-41f8-b34c-afe527684466 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:04.231408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185134a9-ac49-41f8-b34c-afe527684466 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:04.232121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185134a9-ac49-41f8-b34c-afe527684466 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:04.232392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185134a9-ac49-41f8-b34c-afe527684466 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:04.233784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185134a9-ac49-41f8-b34c-afe527684466 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:04.243566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 495/996] 10.222.0.86 () {56 vars in 1122 bytes} [Thu Jul 28 17:54:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:04.243728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640a312-b1ad-4596-9a39-a1357d26bc1c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:04.244489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185134a9-ac49-41f8-b34c-afe527684466 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:04.349801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640a312-b1ad-4596-9a39-a1357d26bc1c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:04.351203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185134a9-ac49-41f8-b34c-afe527684466 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:04.375193 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:04.384854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d5240-cba5-48d9-9af2-4e31f0081798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:04.385532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d5240-cba5-48d9-9af2-4e31f0081798 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:04.385818 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d5240-cba5-48d9-9af2-4e31f0081798 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:04.386224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d5240-cba5-48d9-9af2-4e31f0081798 None admin] PATH_INFO: `/v3/services/image` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:04.386775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d5240-cba5-48d9-9af2-4e31f0081798 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:04.387274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622939e-a2ef-4fbd-81c1-b64ff3bf80a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:04.388437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622939e-a2ef-4fbd-81c1-b64ff3bf80a3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:04.388697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d5240-cba5-48d9-9af2-4e31f0081798 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:04.388805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622939e-a2ef-4fbd-81c1-b64ff3bf80a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:04.388993 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622939e-a2ef-4fbd-81c1-b64ff3bf80a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:04.389937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622939e-a2ef-4fbd-81c1-b64ff3bf80a3 None admin] RBAC: Adding query filter params (interface=admin, service_id=3549fa2830cf41c1ac64c6ee0571f5e5, region_id=RegionOne) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:04.390348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622939e-a2ef-4fbd-81c1-b64ff3bf80a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:04.391863 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622939e-a2ef-4fbd-81c1-b64ff3bf80a3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:04.397077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 496/997] 10.222.0.86 () {56 vars in 1249 bytes} [Thu Jul 28 17:54:04 2022] GET /identity/v3/endpoints?service_id=3549fa2830cf41c1ac64c6ee0571f5e5&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-2e0d5240-cba5-48d9-9af2-4e31f0081798 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:04.397867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:04.398825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:04.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 502/998] 10.222.0.86 () {56 vars in 1105 bytes} [Thu Jul 28 17:54:04 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:04.406094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622939e-a2ef-4fbd-81c1-b64ff3bf80a3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:04.407243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d5240-cba5-48d9-9af2-4e31f0081798 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:04.460904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:04.514219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d5240-cba5-48d9-9af2-4e31f0081798 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:04.514724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622939e-a2ef-4fbd-81c1-b64ff3bf80a3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:04.549862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950fa3d-8143-4435-bd46-abff1253a7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:04.550766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950fa3d-8143-4435-bd46-abff1253a7eb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:04.551195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950fa3d-8143-4435-bd46-abff1253a7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:04.551613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950fa3d-8143-4435-bd46-abff1253a7eb None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:04.552626 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950fa3d-8143-4435-bd46-abff1253a7eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:04.553543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620359e9-e85a-43e1-8722-6f69c152e69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:04.554161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620359e9-e85a-43e1-8722-6f69c152e69c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:04.554447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620359e9-e85a-43e1-8722-6f69c152e69c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:04.554636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950fa3d-8143-4435-bd46-abff1253a7eb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:04.554761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620359e9-e85a-43e1-8722-6f69c152e69c None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:04.555399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620359e9-e85a-43e1-8722-6f69c152e69c None admin] RBAC: Adding query filter params (name=image) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:04.555708 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620359e9-e85a-43e1-8722-6f69c152e69c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:04.557112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620359e9-e85a-43e1-8722-6f69c152e69c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:04.565048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 503/999] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:04.569019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620359e9-e85a-43e1-8722-6f69c152e69c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:04.573330 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 497/1000] 10.222.0.86 () {56 vars in 1087 bytes} [Thu Jul 28 17:54:04 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:54:04.683019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620359e9-e85a-43e1-8722-6f69c152e69c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:04.697800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:04.720519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3688553-8586-4021-bc0d-e38c551e9127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:04.721101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3688553-8586-4021-bc0d-e38c551e9127 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:04.721389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3688553-8586-4021-bc0d-e38c551e9127 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:04.721670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3688553-8586-4021-bc0d-e38c551e9127 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:04.722357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3688553-8586-4021-bc0d-e38c551e9127 None admin] RBAC: Adding query filter params (type=image) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:04.722664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3688553-8586-4021-bc0d-e38c551e9127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:04.724036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3688553-8586-4021-bc0d-e38c551e9127 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:04.729598 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 504/1001] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:05.562695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51de88d-b1d3-4bb1-a701-e46806f57bae None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:05.563220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51de88d-b1d3-4bb1-a701-e46806f57bae None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:05.563220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51de88d-b1d3-4bb1-a701-e46806f57bae None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:05.564743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 498/1002] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:05.569610 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5c838-481f-4a38-b8a3-20880cfee76b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:05.569900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5c838-481f-4a38-b8a3-20880cfee76b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:05.570177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5c838-481f-4a38-b8a3-20880cfee76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:05.588017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2ff5c838-481f-4a38-b8a3-20880cfee76b None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:05.589220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2ff5c838-481f-4a38-b8a3-20880cfee76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:05.659076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 505/1003] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:05.681693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51de88d-b1d3-4bb1-a701-e46806f57bae None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:05.736217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:05.787677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51de88d-b1d3-4bb1-a701-e46806f57bae None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:05.821549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9846509-1275-4284-82fe-ab42310b99fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:05.822191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9846509-1275-4284-82fe-ab42310b99fe None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:05.822472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9846509-1275-4284-82fe-ab42310b99fe None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:05.822743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9846509-1275-4284-82fe-ab42310b99fe None admin] PATH_INFO: `/v3/services/compute` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:05.823374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9846509-1275-4284-82fe-ab42310b99fe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:05.824727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9846509-1275-4284-82fe-ab42310b99fe None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b9846509-1275-4284-82fe-ab42310b99fe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:05.834457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:05.835545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:54:05.836504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:05.841374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 499/1004] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:54:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:05.842518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5c838-481f-4a38-b8a3-20880cfee76b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:05.941166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5c838-481f-4a38-b8a3-20880cfee76b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:05.948739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:05.975692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d389e86-c288-45dc-90ef-ff0b5c8e474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:05.976309 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d389e86-c288-45dc-90ef-ff0b5c8e474c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:05.976589 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d389e86-c288-45dc-90ef-ff0b5c8e474c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:05.976862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d389e86-c288-45dc-90ef-ff0b5c8e474c None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:05.977587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d389e86-c288-45dc-90ef-ff0b5c8e474c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:05.977889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d389e86-c288-45dc-90ef-ff0b5c8e474c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:05.979244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d389e86-c288-45dc-90ef-ff0b5c8e474c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:05.988672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 506/1005] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:54:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:05.989742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9846509-1275-4284-82fe-ab42310b99fe None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:06.092533 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9846509-1275-4284-82fe-ab42310b99fe None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:06.106576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:06.127428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4560f07-8069-4fd3-9ac3-445109835fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:06.128045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4560f07-8069-4fd3-9ac3-445109835fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:06.128325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4560f07-8069-4fd3-9ac3-445109835fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:06.128595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4560f07-8069-4fd3-9ac3-445109835fb0 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:06.129299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4560f07-8069-4fd3-9ac3-445109835fb0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:06.129601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4560f07-8069-4fd3-9ac3-445109835fb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:06.130955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4560f07-8069-4fd3-9ac3-445109835fb0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:06.141039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 500/1006] 10.222.0.86 () {56 vars in 1112 bytes} [Thu Jul 28 17:54:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:06.142229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d389e86-c288-45dc-90ef-ff0b5c8e474c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:06.247450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d389e86-c288-45dc-90ef-ff0b5c8e474c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:06.288653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0d46e-1e47-4eea-b319-fd7d8fd34983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:06.289331 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0d46e-1e47-4eea-b319-fd7d8fd34983 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:06.289614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0d46e-1e47-4eea-b319-fd7d8fd34983 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:06.289894 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0d46e-1e47-4eea-b319-fd7d8fd34983 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:06.290620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0d46e-1e47-4eea-b319-fd7d8fd34983 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:06.292103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0d46e-1e47-4eea-b319-fd7d8fd34983 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:06.382536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 507/1007] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 17:54:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:54:06.589257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea61da9-3bda-4cf9-947b-b48948c600e7 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:06.589763 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea61da9-3bda-4cf9-947b-b48948c600e7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:06.590051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea61da9-3bda-4cf9-947b-b48948c600e7 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:06.591816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 501/1008] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:06.597095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d888eb2-6aef-4d17-9c66-8a8a62ffe136 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:06.597385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d888eb2-6aef-4d17-9c66-8a8a62ffe136 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:06.597659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d888eb2-6aef-4d17-9c66-8a8a62ffe136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:06.615856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1d888eb2-6aef-4d17-9c66-8a8a62ffe136 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:06.617010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1d888eb2-6aef-4d17-9c66-8a8a62ffe136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:06.648339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:06.705892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 508/1009] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:06 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:06.728406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea61da9-3bda-4cf9-947b-b48948c600e7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:06.747981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f59c0c-883a-4868-a7b1-a45cb4d55279 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:06.748270 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f59c0c-883a-4868-a7b1-a45cb4d55279 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:06.748548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f59c0c-883a-4868-a7b1-a45cb4d55279 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:06.750409 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 509/1010] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:54:06.755226 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0403a33-d4da-4058-88cf-d5f9229fbfd9 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:06.755521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0403a33-d4da-4058-88cf-d5f9229fbfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:06.755805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0403a33-d4da-4058-88cf-d5f9229fbfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:06.774691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d0403a33-d4da-4058-88cf-d5f9229fbfd9 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:06.775903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d0403a33-d4da-4058-88cf-d5f9229fbfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:06.840846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea61da9-3bda-4cf9-947b-b48948c600e7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:06.848128 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 510/1011] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:06 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:54:06.871335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0403a33-d4da-4058-88cf-d5f9229fbfd9 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:06.877400 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e4f9b-1833-434b-9407-dadf4eb41af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:06.878046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e4f9b-1833-434b-9407-dadf4eb41af5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:06.878330 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e4f9b-1833-434b-9407-dadf4eb41af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:06.878605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e4f9b-1833-434b-9407-dadf4eb41af5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:06.879253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e4f9b-1833-434b-9407-dadf4eb41af5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:06.880600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e4f9b-1833-434b-9407-dadf4eb41af5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-459e4f9b-1833-434b-9407-dadf4eb41af5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:06.890317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:06.891306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:06.892177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 502/1012] 10.222.0.86 () {56 vars in 1126 bytes} [Thu Jul 28 17:54:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:06.897337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e4f9b-1833-434b-9407-dadf4eb41af5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:06.956935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:06.979682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:06.999431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0403a33-d4da-4058-88cf-d5f9229fbfd9 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:07.024622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e4f9b-1833-434b-9407-dadf4eb41af5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:07.036302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fd37b-f5d1-419d-8ca1-c2407c4c2d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:07.036934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fd37b-f5d1-419d-8ca1-c2407c4c2d7c None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:07.037244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fd37b-f5d1-419d-8ca1-c2407c4c2d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:07.037522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fd37b-f5d1-419d-8ca1-c2407c4c2d7c None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:07.038220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0fd37b-f5d1-419d-8ca1-c2407c4c2d7c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:07.039591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0fd37b-f5d1-419d-8ca1-c2407c4c2d7c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:07.062936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b9c82-e284-46d9-95ae-54adcb288ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:07.063560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9c82-e284-46d9-95ae-54adcb288ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:07.063843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9c82-e284-46d9-95ae-54adcb288ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:07.064118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9c82-e284-46d9-95ae-54adcb288ab0 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:07.064797 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b9c82-e284-46d9-95ae-54adcb288ab0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:07.065135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b9c82-e284-46d9-95ae-54adcb288ab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:07.066486 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b9c82-e284-46d9-95ae-54adcb288ab0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:07.071499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 503/1013] 10.222.0.86 () {56 vars in 1122 bytes} [Thu Jul 28 17:54:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:07.076886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b9c82-e284-46d9-95ae-54adcb288ab0 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:07.113119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 511/1014] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:54:06 2022] POST /identity/v3/services => generated 242 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:54:07.198770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b9c82-e284-46d9-95ae-54adcb288ab0 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:07.208557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:07.247279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15945257-bb35-4cc6-9f05-4e57de87f68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:07.247930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15945257-bb35-4cc6-9f05-4e57de87f68d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:07.248333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15945257-bb35-4cc6-9f05-4e57de87f68d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:07.248613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15945257-bb35-4cc6-9f05-4e57de87f68d None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:07.249603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15945257-bb35-4cc6-9f05-4e57de87f68d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:07.255412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15945257-bb35-4cc6-9f05-4e57de87f68d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:07.255412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15945257-bb35-4cc6-9f05-4e57de87f68d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:07.261174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 504/1015] 10.222.0.86 () {56 vars in 1122 bytes} [Thu Jul 28 17:54:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:07.263095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fd37b-f5d1-419d-8ca1-c2407c4c2d7c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:07.370066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fd37b-f5d1-419d-8ca1-c2407c4c2d7c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:07.405206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba78a4d-2fd3-491a-90da-73b859f0a751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:07.405915 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba78a4d-2fd3-491a-90da-73b859f0a751 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:07.406326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba78a4d-2fd3-491a-90da-73b859f0a751 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:07.406666 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba78a4d-2fd3-491a-90da-73b859f0a751 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:07.407452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba78a4d-2fd3-491a-90da-73b859f0a751 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:07.408882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba78a4d-2fd3-491a-90da-73b859f0a751 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:07.492403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 512/1016] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 17:54:07 2022] POST /identity/v3/endpoints => generated 325 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:08.528808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a83eb-f4c7-4012-aebc-3a73d2a996d9 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:08.529369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a83eb-f4c7-4012-aebc-3a73d2a996d9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:08.529457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a83eb-f4c7-4012-aebc-3a73d2a996d9 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:08.530924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 505/1017] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:08.536005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630576d7-0094-42b2-981f-af5951188288 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:08.536401 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630576d7-0094-42b2-981f-af5951188288 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:08.536562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630576d7-0094-42b2-981f-af5951188288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:08.542908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:08.555125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-630576d7-0094-42b2-981f-af5951188288 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:08.557094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-630576d7-0094-42b2-981f-af5951188288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:08.633535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 513/1018] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:08.657037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a83eb-f4c7-4012-aebc-3a73d2a996d9 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:08.759487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a83eb-f4c7-4012-aebc-3a73d2a996d9 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:08.793440 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f93fa-78a4-4925-adf0-e70c282a0a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:08.794706 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f93fa-78a4-4925-adf0-e70c282a0a39 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:08.794994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f93fa-78a4-4925-adf0-e70c282a0a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:08.795272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f93fa-78a4-4925-adf0-e70c282a0a39 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:08.805479 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f93fa-78a4-4925-adf0-e70c282a0a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b18f93fa-78a4-4925-adf0-e70c282a0a39 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:54:08.810499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:08.811577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:54:08.812524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:08.816695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 506/1019] 10.222.0.86 () {56 vars in 1120 bytes} [Thu Jul 28 17:54:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:08.818075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630576d7-0094-42b2-981f-af5951188288 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:08.916762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630576d7-0094-42b2-981f-af5951188288 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:08.943959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:08.952563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615aaf8-76f6-412b-a487-5d233d5591fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:08.953296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615aaf8-76f6-412b-a487-5d233d5591fc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:08.953647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615aaf8-76f6-412b-a487-5d233d5591fc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:08.953990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615aaf8-76f6-412b-a487-5d233d5591fc None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:08.954783 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615aaf8-76f6-412b-a487-5d233d5591fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:08.955175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615aaf8-76f6-412b-a487-5d233d5591fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:08.956604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615aaf8-76f6-412b-a487-5d233d5591fc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:08.965017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 514/1020] 10.222.0.86 () {56 vars in 1115 bytes} [Thu Jul 28 17:54:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:08.971101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f93fa-78a4-4925-adf0-e70c282a0a39 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.001819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:09.092153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f93fa-78a4-4925-adf0-e70c282a0a39 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.127210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e846c-4e56-4d52-a51d-019e5891927b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.127868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e846c-4e56-4d52-a51d-019e5891927b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.128154 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e846c-4e56-4d52-a51d-019e5891927b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.128434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e846c-4e56-4d52-a51d-019e5891927b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.139521 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e846c-4e56-4d52-a51d-019e5891927b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.140996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e846c-4e56-4d52-a51d-019e5891927b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-4f3e846c-4e56-4d52-a51d-019e5891927b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:54:09.156265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:09.157493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:09.158421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:54:09.158421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:54:09.158421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:09.158421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:09.158669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb6b5a-d8e9-4bf7-bdb8-deb61edbe012 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.159079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb6b5a-d8e9-4bf7-bdb8-deb61edbe012 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.159079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb6b5a-d8e9-4bf7-bdb8-deb61edbe012 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.160676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 515/1022] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:54:09.161039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 507/1022] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:09.163395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb6b5a-d8e9-4bf7-bdb8-deb61edbe012 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.169211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b263d3b-a71b-4b96-a1d5-b68d25870807 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.169211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b263d3b-a71b-4b96-a1d5-b68d25870807 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.169211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b263d3b-a71b-4b96-a1d5-b68d25870807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.184822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9b263d3b-a71b-4b96-a1d5-b68d25870807 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:09.186174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9b263d3b-a71b-4b96-a1d5-b68d25870807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:09.258450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 508/1023] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:54:09.274791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb6b5a-d8e9-4bf7-bdb8-deb61edbe012 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.282017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b263d3b-a71b-4b96-a1d5-b68d25870807 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.298015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:09.311104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc47816-2fce-41bc-96d8-4e6d4510c4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.311721 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc47816-2fce-41bc-96d8-4e6d4510c4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.312796 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc47816-2fce-41bc-96d8-4e6d4510c4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.312796 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc47816-2fce-41bc-96d8-4e6d4510c4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.313018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc47816-2fce-41bc-96d8-4e6d4510c4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:09.313271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc47816-2fce-41bc-96d8-4e6d4510c4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.314624 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc47816-2fce-41bc-96d8-4e6d4510c4d3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.327031 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 516/1024] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:09.333037 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc47816-2fce-41bc-96d8-4e6d4510c4d3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.396453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:09.401970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b263d3b-a71b-4b96-a1d5-b68d25870807 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.446370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e87b7-5e6a-4479-b9a1-7e9243f653b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.447317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e87b7-5e6a-4479-b9a1-7e9243f653b9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.448048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e87b7-5e6a-4479-b9a1-7e9243f653b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.448316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e87b7-5e6a-4479-b9a1-7e9243f653b9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.449178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e87b7-5e6a-4479-b9a1-7e9243f653b9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.451482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e87b7-5e6a-4479-b9a1-7e9243f653b9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.455055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc47816-2fce-41bc-96d8-4e6d4510c4d3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b72e87b7-5e6a-4479-b9a1-7e9243f653b9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:09.461663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:09.462761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:09.463837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 509/1025] 10.222.0.86 () {56 vars in 1105 bytes} [Thu Jul 28 17:54:09 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:09.469076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e87b7-5e6a-4479-b9a1-7e9243f653b9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.496441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15256dd3-1376-4968-95ff-bf2360a6ee9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.497753 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15256dd3-1376-4968-95ff-bf2360a6ee9a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.498098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15256dd3-1376-4968-95ff-bf2360a6ee9a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.498542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15256dd3-1376-4968-95ff-bf2360a6ee9a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.506830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15256dd3-1376-4968-95ff-bf2360a6ee9a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.508471 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15256dd3-1376-4968-95ff-bf2360a6ee9a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-15256dd3-1376-4968-95ff-bf2360a6ee9a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:54:09.521121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:09.527444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 517/1026] 10.222.0.86 () {56 vars in 1093 bytes} [Thu Jul 28 17:54:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20a1ab-6413-4cd8-9cce-651c4206cad4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.528424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20a1ab-6413-4cd8-9cce-651c4206cad4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.529580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20a1ab-6413-4cd8-9cce-651c4206cad4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.529580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 518/1027] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:09.529580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20a1ab-6413-4cd8-9cce-651c4206cad4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.547182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:09.578999 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e87b7-5e6a-4479-b9a1-7e9243f653b9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.611748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04c74a-b873-405d-b3df-c6b5b9be8603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.612362 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04c74a-b873-405d-b3df-c6b5b9be8603 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.612642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04c74a-b873-405d-b3df-c6b5b9be8603 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.612916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04c74a-b873-405d-b3df-c6b5b9be8603 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.613634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04c74a-b873-405d-b3df-c6b5b9be8603 None admin] RBAC: Adding query filter params (name=image) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:09.613941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04c74a-b873-405d-b3df-c6b5b9be8603 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.615292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04c74a-b873-405d-b3df-c6b5b9be8603 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.620042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 510/1028] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:09.621372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002155a-a514-4dd3-87c8-8cf3c6e55957 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.621663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002155a-a514-4dd3-87c8-8cf3c6e55957 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.622471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002155a-a514-4dd3-87c8-8cf3c6e55957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.633790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20a1ab-6413-4cd8-9cce-651c4206cad4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.641469 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7002155a-a514-4dd3-87c8-8cf3c6e55957 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:09.643076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7002155a-a514-4dd3-87c8-8cf3c6e55957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:09.645715 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:09.668504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba571ce-12cc-4005-b92e-de8c1f73f3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.669163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba571ce-12cc-4005-b92e-de8c1f73f3be None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.669390 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba571ce-12cc-4005-b92e-de8c1f73f3be None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.669667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba571ce-12cc-4005-b92e-de8c1f73f3be None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.670467 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba571ce-12cc-4005-b92e-de8c1f73f3be None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:09.670772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba571ce-12cc-4005-b92e-de8c1f73f3be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.672196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba571ce-12cc-4005-b92e-de8c1f73f3be None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.689998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 519/1029] 10.222.0.86 () {56 vars in 1133 bytes} [Thu Jul 28 17:54:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:09.690792 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba571ce-12cc-4005-b92e-de8c1f73f3be None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.714801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 511/1030] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:09.715724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002155a-a514-4dd3-87c8-8cf3c6e55957 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.784596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:09.792018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba571ce-12cc-4005-b92e-de8c1f73f3be None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.819657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002155a-a514-4dd3-87c8-8cf3c6e55957 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.825904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c528a0-cbd6-4085-ab8c-3902ddfee653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.826517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c528a0-cbd6-4085-ab8c-3902ddfee653 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.827120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c528a0-cbd6-4085-ab8c-3902ddfee653 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.827120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c528a0-cbd6-4085-ab8c-3902ddfee653 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.827776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c528a0-cbd6-4085-ab8c-3902ddfee653 None admin] RBAC: Adding query filter params (type=image) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:09.828083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c528a0-cbd6-4085-ab8c-3902ddfee653 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.829523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c528a0-cbd6-4085-ab8c-3902ddfee653 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.834265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 520/1031] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:09.835020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c528a0-cbd6-4085-ab8c-3902ddfee653 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.853327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723216b-6c88-4c56-974c-8cb466c581f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.853986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723216b-6c88-4c56-974c-8cb466c581f6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.854268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723216b-6c88-4c56-974c-8cb466c581f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.854543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723216b-6c88-4c56-974c-8cb466c581f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.864707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:09.923469 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723216b-6c88-4c56-974c-8cb466c581f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.924923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723216b-6c88-4c56-974c-8cb466c581f6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.935801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c528a0-cbd6-4085-ab8c-3902ddfee653 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.978062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435a3c-966f-486b-81fc-88cbc7de247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.978853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435a3c-966f-486b-81fc-88cbc7de247b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.979240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435a3c-966f-486b-81fc-88cbc7de247b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.979637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435a3c-966f-486b-81fc-88cbc7de247b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.981013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14435a3c-966f-486b-81fc-88cbc7de247b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.982964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14435a3c-966f-486b-81fc-88cbc7de247b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.984589 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 521/1032] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:09.985380 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435a3c-966f-486b-81fc-88cbc7de247b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.996386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:10.031833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 512/1033] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:54:09 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:54:10.032753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723216b-6c88-4c56-974c-8cb466c581f6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.083249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:10.086347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435a3c-966f-486b-81fc-88cbc7de247b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.121142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966e504-9643-42ec-8e8b-b5f6bfa4fe5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.122289 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966e504-9643-42ec-8e8b-b5f6bfa4fe5c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.122575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966e504-9643-42ec-8e8b-b5f6bfa4fe5c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.122848 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966e504-9643-42ec-8e8b-b5f6bfa4fe5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.123594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966e504-9643-42ec-8e8b-b5f6bfa4fe5c None admin] RBAC: Adding query filter params (interface=public, service_id=ee88d9367557425d937155ee8e827f42, region_id=RegionOne) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:10.123901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966e504-9643-42ec-8e8b-b5f6bfa4fe5c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.125650 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966e504-9643-42ec-8e8b-b5f6bfa4fe5c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.130620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 522/1034] 10.222.0.86 () {56 vars in 1251 bytes} [Thu Jul 28 17:54:09 2022] GET /identity/v3/endpoints?service_id=ee88d9367557425d937155ee8e827f42&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:10.131327 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966e504-9643-42ec-8e8b-b5f6bfa4fe5c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.138276 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723216b-6c88-4c56-974c-8cb466c581f6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.174391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe9bff-b1fc-4820-b22e-dbc2908d5c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.174941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe9bff-b1fc-4820-b22e-dbc2908d5c67 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.175222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe9bff-b1fc-4820-b22e-dbc2908d5c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.175493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe9bff-b1fc-4820-b22e-dbc2908d5c67 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.176167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe9bff-b1fc-4820-b22e-dbc2908d5c67 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.177617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe9bff-b1fc-4820-b22e-dbc2908d5c67 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.228621 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966e504-9643-42ec-8e8b-b5f6bfa4fe5c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.258676 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 513/1035] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:54:10 2022] POST /identity/v3/projects => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:10.259816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe9bff-b1fc-4820-b22e-dbc2908d5c67 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.262341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35126192-cec4-4a22-b7d9-62e647115e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.263001 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35126192-cec4-4a22-b7d9-62e647115e74 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.263281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35126192-cec4-4a22-b7d9-62e647115e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.264156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35126192-cec4-4a22-b7d9-62e647115e74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.282801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:10.283791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35126192-cec4-4a22-b7d9-62e647115e74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.285337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35126192-cec4-4a22-b7d9-62e647115e74 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-35126192-cec4-4a22-b7d9-62e647115e74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:54:10.302086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:10.303341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:10.304460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:54:10.304460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:54:10.304460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:10.304460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:10.304460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 523/1036] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:10.311225 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35126192-cec4-4a22-b7d9-62e647115e74 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.332791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:10.383973 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe9bff-b1fc-4820-b22e-dbc2908d5c67 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.429286 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a9395-c551-461d-91c0-ef81c640beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.430133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a9395-c551-461d-91c0-ef81c640beed None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.430880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a9395-c551-461d-91c0-ef81c640beed None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.431183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a9395-c551-461d-91c0-ef81c640beed None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.431991 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a9395-c551-461d-91c0-ef81c640beed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.433349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a9395-c551-461d-91c0-ef81c640beed None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.441254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 514/1037] 10.222.0.86 () {56 vars in 1087 bytes} [Thu Jul 28 17:54:10 2022] GET /identity/v3/services => generated 1957 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:54:10.446389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35126192-cec4-4a22-b7d9-62e647115e74 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.493461 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f535bb-5468-41fa-b6b3-73fa8cf50a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.494118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f535bb-5468-41fa-b6b3-73fa8cf50a23 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.494402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f535bb-5468-41fa-b6b3-73fa8cf50a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.495314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f535bb-5468-41fa-b6b3-73fa8cf50a23 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.495434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f535bb-5468-41fa-b6b3-73fa8cf50a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:10.495697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f535bb-5468-41fa-b6b3-73fa8cf50a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.497215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f535bb-5468-41fa-b6b3-73fa8cf50a23 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.511573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 524/1038] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:10.517688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a9395-c551-461d-91c0-ef81c640beed None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.610518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:10.630434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a9395-c551-461d-91c0-ef81c640beed None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.665968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26236875-b562-477a-901d-93d7313119f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.666603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26236875-b562-477a-901d-93d7313119f7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.666888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26236875-b562-477a-901d-93d7313119f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.667184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26236875-b562-477a-901d-93d7313119f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.673934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26236875-b562-477a-901d-93d7313119f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.675439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26236875-b562-477a-901d-93d7313119f7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-26236875-b562-477a-901d-93d7313119f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:54:10.686955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:10.688050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:10.694029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:54:10.694029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:54:10.694029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:54:10.694029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:54:10.694029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:54:10.694029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:54:10.694029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:54:10.694029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:10.694029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 515/1039] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:54:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:10.695784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f535bb-5468-41fa-b6b3-73fa8cf50a23 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.782933 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:10.809121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f535bb-5468-41fa-b6b3-73fa8cf50a23 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.843897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e56e4-364c-40cd-933a-be552f60dda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.844616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e56e4-364c-40cd-933a-be552f60dda6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.844988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e56e4-364c-40cd-933a-be552f60dda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.845339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e56e4-364c-40cd-933a-be552f60dda6 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.846166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e56e4-364c-40cd-933a-be552f60dda6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:10.846550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e56e4-364c-40cd-933a-be552f60dda6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.848061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e56e4-364c-40cd-933a-be552f60dda6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.864878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 525/1040] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:54:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:10.866600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26236875-b562-477a-901d-93d7313119f7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.974199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26236875-b562-477a-901d-93d7313119f7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:11.009348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58358e3-2dad-4f03-bf44-0f2a5e337c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:11.009976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58358e3-2dad-4f03-bf44-0f2a5e337c89 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:11.010263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58358e3-2dad-4f03-bf44-0f2a5e337c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:11.010544 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58358e3-2dad-4f03-bf44-0f2a5e337c89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:11.011368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58358e3-2dad-4f03-bf44-0f2a5e337c89 None admin] RBAC: Adding query filter params (role.id=8e5389a3e81c4e58b716a69b7f74257b, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=7d7bcb4e7c434622b650cdfc500a9680) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:11.011684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58358e3-2dad-4f03-bf44-0f2a5e337c89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:11.013704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58358e3-2dad-4f03-bf44-0f2a5e337c89 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:11.025011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 516/1041] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:54:10 2022] GET /identity/v3/role_assignments?user.id=7d7bcb4e7c434622b650cdfc500a9680&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=8e5389a3e81c4e58b716a69b7f74257b => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:12.363614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b8160-7c30-4f6f-b07f-02e5aac816ee None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:12.364163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b8160-7c30-4f6f-b07f-02e5aac816ee None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:12.364163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b8160-7c30-4f6f-b07f-02e5aac816ee None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:12.365597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 526/1042] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:12.370604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa35541-9de4-4943-9f62-28891eda4ae7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:12.371001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa35541-9de4-4943-9f62-28891eda4ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:12.371164 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa35541-9de4-4943-9f62-28891eda4ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:12.384332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:12.390954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-2fa35541-9de4-4943-9f62-28891eda4ae7 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:12.392147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2fa35541-9de4-4943-9f62-28891eda4ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:12.466991 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 517/1043] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:12.490945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b8160-7c30-4f6f-b07f-02e5aac816ee None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:12.492882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00de958-37f9-4f1c-91fd-91d58c499320 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:12.493226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00de958-37f9-4f1c-91fd-91d58c499320 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:12.493512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00de958-37f9-4f1c-91fd-91d58c499320 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:12.494881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 518/1044] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:12.502980 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4793a4-f2e0-4fce-b2f4-ba04cf755566 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:12.503511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4793a4-f2e0-4fce-b2f4-ba04cf755566 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:12.503982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4793a4-f2e0-4fce-b2f4-ba04cf755566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:12.523733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5d4793a4-f2e0-4fce-b2f4-ba04cf755566 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:12.524921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5d4793a4-f2e0-4fce-b2f4-ba04cf755566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:12.596800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 519/1045] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:54:12.602930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b8160-7c30-4f6f-b07f-02e5aac816ee None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:12.618984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4793a4-f2e0-4fce-b2f4-ba04cf755566 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:12.640139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee2450-d4b8-42a7-97de-689f510e465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:12.641217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee2450-d4b8-42a7-97de-689f510e465a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:12.641705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee2450-d4b8-42a7-97de-689f510e465a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:12.642173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee2450-d4b8-42a7-97de-689f510e465a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:12.643503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee2450-d4b8-42a7-97de-689f510e465a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:12.645135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee2450-d4b8-42a7-97de-689f510e465a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:12.651743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 527/1046] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:12.652501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee2450-d4b8-42a7-97de-689f510e465a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:12.656140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:12.691866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:12.728196 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4793a4-f2e0-4fce-b2f4-ba04cf755566 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:12.762805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee2450-d4b8-42a7-97de-689f510e465a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:12.769689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de95e78-6722-47ce-b306-728c555cc576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:12.770826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de95e78-6722-47ce-b306-728c555cc576 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:12.771120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de95e78-6722-47ce-b306-728c555cc576 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:12.771405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de95e78-6722-47ce-b306-728c555cc576 None admin] PATH_INFO: `/v3/services/image` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:12.772618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de95e78-6722-47ce-b306-728c555cc576 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:12.774351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de95e78-6722-47ce-b306-728c555cc576 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-2de95e78-6722-47ce-b306-728c555cc576 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:12.784182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:12.785212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:12.789892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 520/1047] 10.222.0.86 () {56 vars in 1105 bytes} [Thu Jul 28 17:54:12 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:12.791647 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de95e78-6722-47ce-b306-728c555cc576 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:12.803594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c54389-024c-4f04-9938-ea771f37b436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:12.804245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c54389-024c-4f04-9938-ea771f37b436 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:12.804551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c54389-024c-4f04-9938-ea771f37b436 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:12.804814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c54389-024c-4f04-9938-ea771f37b436 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:12.805547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c54389-024c-4f04-9938-ea771f37b436 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:12.806977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c54389-024c-4f04-9938-ea771f37b436 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:12.876481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 528/1048] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:54:12 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:54:12.900915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de95e78-6722-47ce-b306-728c555cc576 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:12.934768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:12.941881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67affdc-da5a-4194-8df1-34689025e448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:12.942530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67affdc-da5a-4194-8df1-34689025e448 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:12.942813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67affdc-da5a-4194-8df1-34689025e448 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:12.943088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67affdc-da5a-4194-8df1-34689025e448 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:12.943788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67affdc-da5a-4194-8df1-34689025e448 None admin] RBAC: Adding query filter params (name=image) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:12.944239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67affdc-da5a-4194-8df1-34689025e448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:12.945650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67affdc-da5a-4194-8df1-34689025e448 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:12.956158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 521/1049] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:12.957576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c54389-024c-4f04-9938-ea771f37b436 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:13.031406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:13.071812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3abaf-f968-431c-91b4-b137286014a6 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:13.072105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3abaf-f968-431c-91b4-b137286014a6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:13.072381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3abaf-f968-431c-91b4-b137286014a6 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:13.074002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 522/1050] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:13.078279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c54389-024c-4f04-9938-ea771f37b436 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:13.079586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d9464-e5d5-4d40-acca-bff57f52b6c8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:13.079950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d9464-e5d5-4d40-acca-bff57f52b6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:13.080306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d9464-e5d5-4d40-acca-bff57f52b6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:13.105588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5d3d9464-e5d5-4d40-acca-bff57f52b6c8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:13.107335 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5d3d9464-e5d5-4d40-acca-bff57f52b6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:13.124636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55df672-5ec7-43ae-b366-7a651dd60996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:13.125724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55df672-5ec7-43ae-b366-7a651dd60996 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:13.126015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55df672-5ec7-43ae-b366-7a651dd60996 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:13.126300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55df672-5ec7-43ae-b366-7a651dd60996 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:13.127056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55df672-5ec7-43ae-b366-7a651dd60996 None admin] RBAC: Adding query filter params (type=image) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:13.127306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55df672-5ec7-43ae-b366-7a651dd60996 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:13.129176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55df672-5ec7-43ae-b366-7a651dd60996 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:13.134308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 529/1051] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:13.140156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55df672-5ec7-43ae-b366-7a651dd60996 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:13.184122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 523/1052] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:54:13.208182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d9464-e5d5-4d40-acca-bff57f52b6c8 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:13.248490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55df672-5ec7-43ae-b366-7a651dd60996 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:13.273858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:13.285869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508c0f2-caaa-4953-a738-22ce6b98e9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:13.286494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508c0f2-caaa-4953-a738-22ce6b98e9fc None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:13.286774 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508c0f2-caaa-4953-a738-22ce6b98e9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:13.287047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508c0f2-caaa-4953-a738-22ce6b98e9fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:13.287756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508c0f2-caaa-4953-a738-22ce6b98e9fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:13.289250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508c0f2-caaa-4953-a738-22ce6b98e9fc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:13.316009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d9464-e5d5-4d40-acca-bff57f52b6c8 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:13.338994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:13.350299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e85c9f-cda7-4f24-a9a3-3f8c94e992f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:13.350995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85c9f-cda7-4f24-a9a3-3f8c94e992f7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:13.351278 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85c9f-cda7-4f24-a9a3-3f8c94e992f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:13.351557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85c9f-cda7-4f24-a9a3-3f8c94e992f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:13.361965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e85c9f-cda7-4f24-a9a3-3f8c94e992f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-13e85c9f-cda7-4f24-a9a3-3f8c94e992f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:54:13.366831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:13.367952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:54:13.369023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:13.373353 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 524/1053] 10.222.0.86 () {56 vars in 1120 bytes} [Thu Jul 28 17:54:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:13.374529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e85c9f-cda7-4f24-a9a3-3f8c94e992f7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:13.389542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 530/1054] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 17:54:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:54:13.496883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e85c9f-cda7-4f24-a9a3-3f8c94e992f7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:13.545961 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a0263-a34b-437b-bc3d-788421e2febd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:13.546591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a0263-a34b-437b-bc3d-788421e2febd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:13.546877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a0263-a34b-437b-bc3d-788421e2febd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:13.547342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a0263-a34b-437b-bc3d-788421e2febd None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:13.548068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a0263-a34b-437b-bc3d-788421e2febd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:13.548374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a0263-a34b-437b-bc3d-788421e2febd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:13.549967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a0263-a34b-437b-bc3d-788421e2febd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:13.560506 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 525/1055] 10.222.0.86 () {56 vars in 1115 bytes} [Thu Jul 28 17:54:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:13.566511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508c0f2-caaa-4953-a738-22ce6b98e9fc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:13.676439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508c0f2-caaa-4953-a738-22ce6b98e9fc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:13.710525 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d57e7-ab04-407d-9bad-312cab8a58cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:13.711172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d57e7-ab04-407d-9bad-312cab8a58cb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:13.711460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d57e7-ab04-407d-9bad-312cab8a58cb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:13.711742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d57e7-ab04-407d-9bad-312cab8a58cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:13.723847 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d57e7-ab04-407d-9bad-312cab8a58cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:13.725325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d57e7-ab04-407d-9bad-312cab8a58cb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a06d57e7-ab04-407d-9bad-312cab8a58cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:54:13.739612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:13.740575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:13.741568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:54:13.741568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:54:13.741568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:13.741568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:13.741568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 531/1056] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:13.747637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a0263-a34b-437b-bc3d-788421e2febd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:13.792398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:13.850537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a0263-a34b-437b-bc3d-788421e2febd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:13.887240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30882e1c-5ede-4910-8d70-955a8d9b698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:13.887869 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30882e1c-5ede-4910-8d70-955a8d9b698e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:13.888149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30882e1c-5ede-4910-8d70-955a8d9b698e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:13.888429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30882e1c-5ede-4910-8d70-955a8d9b698e None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:13.889156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30882e1c-5ede-4910-8d70-955a8d9b698e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:13.889467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30882e1c-5ede-4910-8d70-955a8d9b698e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:13.890838 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30882e1c-5ede-4910-8d70-955a8d9b698e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:13.907550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 526/1057] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:13.908743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d57e7-ab04-407d-9bad-312cab8a58cb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:13.940825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:14.011307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d57e7-ab04-407d-9bad-312cab8a58cb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:14.045939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8db14-25d6-4cf5-a6ae-f122d0624643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:14.046751 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8db14-25d6-4cf5-a6ae-f122d0624643 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:14.047156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8db14-25d6-4cf5-a6ae-f122d0624643 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:14.047539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8db14-25d6-4cf5-a6ae-f122d0624643 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:14.055514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8db14-25d6-4cf5-a6ae-f122d0624643 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:14.057110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8db14-25d6-4cf5-a6ae-f122d0624643 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-93b8db14-25d6-4cf5-a6ae-f122d0624643 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:54:14.068895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:14.070117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:14.071322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:14.071322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:14.071322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:54:14.071322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:54:14.071322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:54:14.071322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:54:14.071322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:54:14.071322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:54:14.071322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:54:14.071322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:14.077533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 532/1058] 10.222.0.86 () {56 vars in 1093 bytes} [Thu Jul 28 17:54:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:14.078736 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30882e1c-5ede-4910-8d70-955a8d9b698e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:14.185328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30882e1c-5ede-4910-8d70-955a8d9b698e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:14.186868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:14.220890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894293a-2311-4064-a9d1-0354235f6293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:14.221550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894293a-2311-4064-a9d1-0354235f6293 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:14.221903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894293a-2311-4064-a9d1-0354235f6293 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:14.222250 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894293a-2311-4064-a9d1-0354235f6293 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:14.223115 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894293a-2311-4064-a9d1-0354235f6293 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:14.223501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894293a-2311-4064-a9d1-0354235f6293 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:14.225011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894293a-2311-4064-a9d1-0354235f6293 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:14.242742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 527/1059] 10.222.0.86 () {56 vars in 1133 bytes} [Thu Jul 28 17:54:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:14.248199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8db14-25d6-4cf5-a6ae-f122d0624643 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:14.339312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:14.348746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8db14-25d6-4cf5-a6ae-f122d0624643 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:14.382678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa01ed-7e37-45b6-8f85-c6e3ed2d5586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:14.383322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa01ed-7e37-45b6-8f85-c6e3ed2d5586 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:14.383606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa01ed-7e37-45b6-8f85-c6e3ed2d5586 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:14.383882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa01ed-7e37-45b6-8f85-c6e3ed2d5586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:14.395020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa01ed-7e37-45b6-8f85-c6e3ed2d5586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:14.396492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa01ed-7e37-45b6-8f85-c6e3ed2d5586 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-aafa01ed-7e37-45b6-8f85-c6e3ed2d5586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:54:14.411335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:14.412278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:14.413249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:54:14.413249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:54:14.413249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:14.413249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:14.419117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 533/1060] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:14.419279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894293a-2311-4064-a9d1-0354235f6293 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:14.526634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894293a-2311-4064-a9d1-0354235f6293 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:14.563998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4ee18-73d4-4f61-a736-7b678957d2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:14.564625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4ee18-73d4-4f61-a736-7b678957d2db None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:14.564911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4ee18-73d4-4f61-a736-7b678957d2db None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:14.565216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4ee18-73d4-4f61-a736-7b678957d2db None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:14.565916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4ee18-73d4-4f61-a736-7b678957d2db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:14.566223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4ee18-73d4-4f61-a736-7b678957d2db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:14.567582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4ee18-73d4-4f61-a736-7b678957d2db None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:14.583578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 528/1061] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:14.585059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa01ed-7e37-45b6-8f85-c6e3ed2d5586 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:14.688948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa01ed-7e37-45b6-8f85-c6e3ed2d5586 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:14.722725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd4333-378a-4248-8340-3cb07703350a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:14.723512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd4333-378a-4248-8340-3cb07703350a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:14.723898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd4333-378a-4248-8340-3cb07703350a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:14.724280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd4333-378a-4248-8340-3cb07703350a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:14.731556 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd4333-378a-4248-8340-3cb07703350a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:14.733133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd4333-378a-4248-8340-3cb07703350a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-adfd4333-378a-4248-8340-3cb07703350a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:54:14.744451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:14.745573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:14.747231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:54:14.747231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:54:14.747231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:54:14.747231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:54:14.747231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:54:14.747231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:54:14.747231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:54:14.747231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:14.748801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 534/1062] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:54:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:14.754641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4ee18-73d4-4f61-a736-7b678957d2db None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:14.777840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:14.856879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4ee18-73d4-4f61-a736-7b678957d2db None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:14.898413 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828f084-e896-4152-b121-e3695067f860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:14.899219 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f084-e896-4152-b121-e3695067f860 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:14.901105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f084-e896-4152-b121-e3695067f860 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:14.901507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f084-e896-4152-b121-e3695067f860 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:14.902392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828f084-e896-4152-b121-e3695067f860 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:14.902845 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828f084-e896-4152-b121-e3695067f860 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:14.906024 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828f084-e896-4152-b121-e3695067f860 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:14.925135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 529/1063] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:54:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:14.926494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd4333-378a-4248-8340-3cb07703350a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:14.930517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:14.992128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ada234-cdbf-40b5-9df3-5fb18e64ce4d None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:14.992412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ada234-cdbf-40b5-9df3-5fb18e64ce4d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:14.992690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ada234-cdbf-40b5-9df3-5fb18e64ce4d None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:14.994235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 530/1064] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:54:15.001517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a380-ee82-4225-afec-e7af4ce8846d None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.002741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a380-ee82-4225-afec-e7af4ce8846d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.003582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a380-ee82-4225-afec-e7af4ce8846d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.024740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-cdf4a380-ee82-4225-afec-e7af4ce8846d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:15.026243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cdf4a380-ee82-4225-afec-e7af4ce8846d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:15.040669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd4333-378a-4248-8340-3cb07703350a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:15.078308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99b19f-bc0f-4afd-90df-41e2b65e30a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:15.079162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99b19f-bc0f-4afd-90df-41e2b65e30a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.079552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99b19f-bc0f-4afd-90df-41e2b65e30a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.079934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99b19f-bc0f-4afd-90df-41e2b65e30a2 None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/7d7bcb4e7c434622b650cdfc500a9680/roles/8e5389a3e81c4e58b716a69b7f74257b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.082805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99b19f-bc0f-4afd-90df-41e2b65e30a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=7d7bcb4e7c434622b650cdfc500a9680, role_id=8e5389a3e81c4e58b716a69b7f74257b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:15.084740 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99b19f-bc0f-4afd-90df-41e2b65e30a2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:15.111178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 531/1065] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:54:15.134548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a380-ee82-4225-afec-e7af4ce8846d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:15.146672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:15.159263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 535/1066] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:54:14 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/7d7bcb4e7c434622b650cdfc500a9680/roles/8e5389a3e81c4e58b716a69b7f74257b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:15.254997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a380-ee82-4225-afec-e7af4ce8846d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:15.294060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c93a6e-d221-4b77-82f8-676c2fd99f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:15.294966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c93a6e-d221-4b77-82f8-676c2fd99f7f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.295431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c93a6e-d221-4b77-82f8-676c2fd99f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.295857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c93a6e-d221-4b77-82f8-676c2fd99f7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.307791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c93a6e-d221-4b77-82f8-676c2fd99f7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-f0c93a6e-d221-4b77-82f8-676c2fd99f7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:54:15.313072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:15.314039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:15.314963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 532/1067] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:54:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:15.320902 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99b19f-bc0f-4afd-90df-41e2b65e30a2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:15.392121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:15.438314 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38b1d2-2a92-499f-97a5-559ce033ec65 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.438801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38b1d2-2a92-499f-97a5-559ce033ec65 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.438903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38b1d2-2a92-499f-97a5-559ce033ec65 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.439230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99b19f-bc0f-4afd-90df-41e2b65e30a2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:15.440345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 533/1068] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:54:15.445095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf9793-54e2-4315-b4e5-172c81fa7566 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.445386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf9793-54e2-4315-b4e5-172c81fa7566 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.448014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf9793-54e2-4315-b4e5-172c81fa7566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.463908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1caf9793-54e2-4315-b4e5-172c81fa7566 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:15.465123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1caf9793-54e2-4315-b4e5-172c81fa7566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:15.476105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db64e0a-8d3e-4237-af49-7856b32353b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:15.476759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64e0a-8d3e-4237-af49-7856b32353b1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.477165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64e0a-8d3e-4237-af49-7856b32353b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.477444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64e0a-8d3e-4237-af49-7856b32353b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.478167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db64e0a-8d3e-4237-af49-7856b32353b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:15.478477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db64e0a-8d3e-4237-af49-7856b32353b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:15.479963 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db64e0a-8d3e-4237-af49-7856b32353b1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:15.494718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 536/1069] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:54:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:15.495762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db64e0a-8d3e-4237-af49-7856b32353b1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:15.534557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:15.539279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 534/1070] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:54:15.565935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf9793-54e2-4315-b4e5-172c81fa7566 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:15.608547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db64e0a-8d3e-4237-af49-7856b32353b1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:15.637713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:15.646620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72110ea5-03a9-4504-8e88-d354275b68d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:15.647263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72110ea5-03a9-4504-8e88-d354275b68d8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.647541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72110ea5-03a9-4504-8e88-d354275b68d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.647989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72110ea5-03a9-4504-8e88-d354275b68d8 None admin] PATH_INFO: `/v3/users/9346bf9dcb3c4433861b9f912a811807` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.662085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72110ea5-03a9-4504-8e88-d354275b68d8 None admin] RBAC: Authorizing `identity:get_user(user_id=9346bf9dcb3c4433861b9f912a811807)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:15.663518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72110ea5-03a9-4504-8e88-d354275b68d8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:15.665726 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 537/1071] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:15 2022] GET /identity/v3/users/9346bf9dcb3c4433861b9f912a811807 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:15.670903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72110ea5-03a9-4504-8e88-d354275b68d8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:15.676521 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf9793-54e2-4315-b4e5-172c81fa7566 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:15.712332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0cb96-509f-4563-8a99-bda8a648f438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:15.713008 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cb96-509f-4563-8a99-bda8a648f438 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.713295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cb96-509f-4563-8a99-bda8a648f438 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.713567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cb96-509f-4563-8a99-bda8a648f438 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.782305 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72110ea5-03a9-4504-8e88-d354275b68d8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:15.785407 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:15.787901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0cb96-509f-4563-8a99-bda8a648f438 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:15.789473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0cb96-509f-4563-8a99-bda8a648f438 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:15.822853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bbc80-ffe2-4a54-9e4d-3a7157bd6cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:15.823407 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bbc80-ffe2-4a54-9e4d-3a7157bd6cee None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:15.823689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bbc80-ffe2-4a54-9e4d-3a7157bd6cee None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:15.823964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bbc80-ffe2-4a54-9e4d-3a7157bd6cee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:15.884842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:15.898191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 535/1072] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:54:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:54:15.899840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0cb96-509f-4563-8a99-bda8a648f438 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:15.900255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bbc80-ffe2-4a54-9e4d-3a7157bd6cee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:15.901882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bbc80-ffe2-4a54-9e4d-3a7157bd6cee None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:16.003668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:16.006896 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0cb96-509f-4563-8a99-bda8a648f438 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:16.011126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 538/1073] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:54:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:54:16.011979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bbc80-ffe2-4a54-9e4d-3a7157bd6cee None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:16.042082 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4aa3d-c83c-42d2-8ae0-6035fdd9ccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:16.042841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4aa3d-c83c-42d2-8ae0-6035fdd9ccf8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:16.043125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4aa3d-c83c-42d2-8ae0-6035fdd9ccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:16.043932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4aa3d-c83c-42d2-8ae0-6035fdd9ccf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:16.055635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4aa3d-c83c-42d2-8ae0-6035fdd9ccf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:16.057212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4aa3d-c83c-42d2-8ae0-6035fdd9ccf8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-2af4aa3d-c83c-42d2-8ae0-6035fdd9ccf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:54:16.073936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:16.074908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:16.075901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:54:16.075901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:54:16.075901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:16.075901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:16.075901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 536/1074] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:16.081316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4aa3d-c83c-42d2-8ae0-6035fdd9ccf8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:16.100722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:16.118350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bbc80-ffe2-4a54-9e4d-3a7157bd6cee None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:16.157318 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11a938-7fe4-4974-9414-c8a9919bd489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:16.157969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11a938-7fe4-4974-9414-c8a9919bd489 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:16.158258 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11a938-7fe4-4974-9414-c8a9919bd489 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:16.158591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11a938-7fe4-4974-9414-c8a9919bd489 None admin] PATH_INFO: `/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:16.159756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11a938-7fe4-4974-9414-c8a9919bd489 None admin] RBAC: Authorizing `identity:get_project(project_id=7acc94439e334b88b0c9ae0cb8afc6e0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:16.161328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11a938-7fe4-4974-9414-c8a9919bd489 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:16.168896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 539/1075] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:54:16 2022] GET /identity/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:16.169520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11a938-7fe4-4974-9414-c8a9919bd489 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:16.190525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4aa3d-c83c-42d2-8ae0-6035fdd9ccf8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:16.227038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9c9ed-6623-48e1-a3a2-081ffb7e63e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:16.227668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9c9ed-6623-48e1-a3a2-081ffb7e63e4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:16.227956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9c9ed-6623-48e1-a3a2-081ffb7e63e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:16.228233 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9c9ed-6623-48e1-a3a2-081ffb7e63e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:16.229012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9c9ed-6623-48e1-a3a2-081ffb7e63e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:16.229332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9c9ed-6623-48e1-a3a2-081ffb7e63e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:16.235402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9c9ed-6623-48e1-a3a2-081ffb7e63e4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:16.253005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 537/1076] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:16.290888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11a938-7fe4-4974-9414-c8a9919bd489 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:16.330458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77362d4-5846-47af-9bed-0743775e6e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:16.331004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77362d4-5846-47af-9bed-0743775e6e0a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:16.331287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77362d4-5846-47af-9bed-0743775e6e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:16.331564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77362d4-5846-47af-9bed-0743775e6e0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:16.332377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77362d4-5846-47af-9bed-0743775e6e0a None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=7acc94439e334b88b0c9ae0cb8afc6e0, user.id=9346bf9dcb3c4433861b9f912a811807) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:16.332689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77362d4-5846-47af-9bed-0743775e6e0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:16.334267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77362d4-5846-47af-9bed-0743775e6e0a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:16.349013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 540/1077] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:54:16 2022] GET /identity/v3/role_assignments?user.id=9346bf9dcb3c4433861b9f912a811807&scope.project.id=7acc94439e334b88b0c9ae0cb8afc6e0&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:17.234912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a67d-d524-444e-8801-c78b46a52b9d None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:17.235470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a67d-d524-444e-8801-c78b46a52b9d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:17.235470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a67d-d524-444e-8801-c78b46a52b9d None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:17.236838 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 538/1078] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:17.242087 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041b12-4efb-497d-9157-ea7851c6b359 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:17.242499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041b12-4efb-497d-9157-ea7851c6b359 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:17.242639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041b12-4efb-497d-9157-ea7851c6b359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:17.261004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-15041b12-4efb-497d-9157-ea7851c6b359 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:17.262797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-15041b12-4efb-497d-9157-ea7851c6b359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:17.273238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:17.328265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 541/1079] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:17.351336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a67d-d524-444e-8801-c78b46a52b9d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:17.366477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:17.458419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a67d-d524-444e-8801-c78b46a52b9d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:17.493511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dec54-69d8-4682-97af-0ed1c14ba364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:17.494076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dec54-69d8-4682-97af-0ed1c14ba364 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:17.494359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dec54-69d8-4682-97af-0ed1c14ba364 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:17.494635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dec54-69d8-4682-97af-0ed1c14ba364 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:17.505742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979dec54-69d8-4682-97af-0ed1c14ba364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-979dec54-69d8-4682-97af-0ed1c14ba364 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:54:17.510628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:17.511577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:17.512711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 539/1080] 10.222.0.86 () {56 vars in 1120 bytes} [Thu Jul 28 17:54:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:17.517956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15041b12-4efb-497d-9157-ea7851c6b359 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:17.615324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15041b12-4efb-497d-9157-ea7851c6b359 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:17.666931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af096964-13d1-4022-808f-9d8decece59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:17.667565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af096964-13d1-4022-808f-9d8decece59b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:17.667849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af096964-13d1-4022-808f-9d8decece59b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:17.668129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af096964-13d1-4022-808f-9d8decece59b None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:17.669487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af096964-13d1-4022-808f-9d8decece59b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:17.669794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af096964-13d1-4022-808f-9d8decece59b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:17.671162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af096964-13d1-4022-808f-9d8decece59b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:17.679456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 542/1081] 10.222.0.86 () {56 vars in 1115 bytes} [Thu Jul 28 17:54:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:17.684846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dec54-69d8-4682-97af-0ed1c14ba364 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:17.757247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:17.785687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dec54-69d8-4682-97af-0ed1c14ba364 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:17.819559 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018fced-4a8c-4c2b-b1b7-556459a9b438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:17.820190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018fced-4a8c-4c2b-b1b7-556459a9b438 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:17.820469 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018fced-4a8c-4c2b-b1b7-556459a9b438 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:17.820739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018fced-4a8c-4c2b-b1b7-556459a9b438 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:17.830558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018fced-4a8c-4c2b-b1b7-556459a9b438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:17.831979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018fced-4a8c-4c2b-b1b7-556459a9b438 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0018fced-4a8c-4c2b-b1b7-556459a9b438 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:54:17.846616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:17.847681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:17.848826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:54:17.848826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:54:17.848826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:17.848826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:17.852411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 540/1082] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:17.853585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af096964-13d1-4022-808f-9d8decece59b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:17.865713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:17.952950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af096964-13d1-4022-808f-9d8decece59b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:17.989288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5371f-eb73-48ac-809a-210953c2bff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:17.989914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5371f-eb73-48ac-809a-210953c2bff7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:17.990199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5371f-eb73-48ac-809a-210953c2bff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:17.990478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5371f-eb73-48ac-809a-210953c2bff7 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:17.991188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5371f-eb73-48ac-809a-210953c2bff7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:17.991497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5371f-eb73-48ac-809a-210953c2bff7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:17.992875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5371f-eb73-48ac-809a-210953c2bff7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:18.004953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 543/1083] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:18.010729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018fced-4a8c-4c2b-b1b7-556459a9b438 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:18.109694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018fced-4a8c-4c2b-b1b7-556459a9b438 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:18.133083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:18.144081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d8c9d-6722-4bad-9211-b587f1343476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:18.144723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d8c9d-6722-4bad-9211-b587f1343476 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:18.145053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d8c9d-6722-4bad-9211-b587f1343476 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:18.145309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d8c9d-6722-4bad-9211-b587f1343476 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:18.154106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d8c9d-6722-4bad-9211-b587f1343476 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:18.155533 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d8c9d-6722-4bad-9211-b587f1343476 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-873d8c9d-6722-4bad-9211-b587f1343476 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:54:18.166883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:18.167880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:18.168850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:18.168850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:18.168850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:54:18.168850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:54:18.168850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:54:18.168850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:54:18.168850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:54:18.168850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:54:18.168850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:54:18.168850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:18.168850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 541/1084] 10.222.0.86 () {56 vars in 1093 bytes} [Thu Jul 28 17:54:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:18.173813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5371f-eb73-48ac-809a-210953c2bff7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:18.239487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:18.272381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5371f-eb73-48ac-809a-210953c2bff7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:18.306172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913c752-aa4d-433a-b074-28b466c5dc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:18.306792 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c752-aa4d-433a-b074-28b466c5dc63 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:18.307077 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c752-aa4d-433a-b074-28b466c5dc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:18.307358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c752-aa4d-433a-b074-28b466c5dc63 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:18.308156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913c752-aa4d-433a-b074-28b466c5dc63 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:18.308470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913c752-aa4d-433a-b074-28b466c5dc63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:18.309963 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913c752-aa4d-433a-b074-28b466c5dc63 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:18.327334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 544/1085] 10.222.0.86 () {56 vars in 1133 bytes} [Thu Jul 28 17:54:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:18.332994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d8c9d-6722-4bad-9211-b587f1343476 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:18.344083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f370f44-4105-4732-a3d9-1f1ad66e048e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:18.344369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f370f44-4105-4732-a3d9-1f1ad66e048e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:18.344647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f370f44-4105-4732-a3d9-1f1ad66e048e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:18.346179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 545/1086] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:54:18.350937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ece098-430c-4101-ac3c-3678bfdd39c0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:18.351224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ece098-430c-4101-ac3c-3678bfdd39c0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:18.351505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ece098-430c-4101-ac3c-3678bfdd39c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:18.370217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-57ece098-430c-4101-ac3c-3678bfdd39c0 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:18.371470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-57ece098-430c-4101-ac3c-3678bfdd39c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:18.435724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d8c9d-6722-4bad-9211-b587f1343476 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:18.443585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 546/1087] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:54:18.466108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ece098-430c-4101-ac3c-3678bfdd39c0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:18.470076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0f087-1e78-440f-a5d2-c2d8efe28499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:18.470679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0f087-1e78-440f-a5d2-c2d8efe28499 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:18.470957 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0f087-1e78-440f-a5d2-c2d8efe28499 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:18.471230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0f087-1e78-440f-a5d2-c2d8efe28499 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:18.519788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:18.522714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:18.538479 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0f087-1e78-440f-a5d2-c2d8efe28499 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:18.539929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0f087-1e78-440f-a5d2-c2d8efe28499 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:18.562492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ece098-430c-4101-ac3c-3678bfdd39c0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:18.597492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a322aba-5694-4262-a789-510214d47e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:18.598131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a322aba-5694-4262-a789-510214d47e66 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:18.598414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a322aba-5694-4262-a789-510214d47e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:18.598692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a322aba-5694-4262-a789-510214d47e66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:18.609203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a322aba-5694-4262-a789-510214d47e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-5a322aba-5694-4262-a789-510214d47e66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:54:18.614085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:18.615117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:18.616332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 547/1088] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:54:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:18.621091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a322aba-5694-4262-a789-510214d47e66 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:18.641219 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 542/1089] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:54:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:54:18.646722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0f087-1e78-440f-a5d2-c2d8efe28499 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:18.722882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a322aba-5694-4262-a789-510214d47e66 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:18.729249 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:18.739994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:18.745546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0f087-1e78-440f-a5d2-c2d8efe28499 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:18.757534 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec6944-af8a-4dd2-ac25-09f382d26d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:18.758152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec6944-af8a-4dd2-ac25-09f382d26d6e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:18.758433 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec6944-af8a-4dd2-ac25-09f382d26d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:18.758707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec6944-af8a-4dd2-ac25-09f382d26d6e None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:18.759422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec6944-af8a-4dd2-ac25-09f382d26d6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:18.759727 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec6944-af8a-4dd2-ac25-09f382d26d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:18.761260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec6944-af8a-4dd2-ac25-09f382d26d6e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:18.769321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 548/1090] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:54:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:18.775074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec6944-af8a-4dd2-ac25-09f382d26d6e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:18.780539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d099229-96f1-4b19-91f5-b75a000131b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:18.781217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d099229-96f1-4b19-91f5-b75a000131b5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:18.781504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d099229-96f1-4b19-91f5-b75a000131b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:18.781798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d099229-96f1-4b19-91f5-b75a000131b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:18.792126 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d099229-96f1-4b19-91f5-b75a000131b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:18.794167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d099229-96f1-4b19-91f5-b75a000131b5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-2d099229-96f1-4b19-91f5-b75a000131b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:54:18.808390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:18.809340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:18.810248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:54:18.810248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:54:18.810248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:54:18.810248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:18.810248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 543/1091] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:18.815106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d099229-96f1-4b19-91f5-b75a000131b5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:18.874347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec6944-af8a-4dd2-ac25-09f382d26d6e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:18.909563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219589bc-b729-43d7-8a6d-6d7762e971ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:18.910197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219589bc-b729-43d7-8a6d-6d7762e971ec None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:18.910472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219589bc-b729-43d7-8a6d-6d7762e971ec None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:18.910746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219589bc-b729-43d7-8a6d-6d7762e971ec None admin] PATH_INFO: `/v3/users/9346bf9dcb3c4433861b9f912a811807` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:18.911916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219589bc-b729-43d7-8a6d-6d7762e971ec None admin] RBAC: Authorizing `identity:get_user(user_id=9346bf9dcb3c4433861b9f912a811807)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:18.913364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219589bc-b729-43d7-8a6d-6d7762e971ec None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:18.914415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d099229-96f1-4b19-91f5-b75a000131b5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:18.914962 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 549/1092] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:18 2022] GET /identity/v3/users/9346bf9dcb3c4433861b9f912a811807 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:18.920768 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219589bc-b729-43d7-8a6d-6d7762e971ec None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:18.947787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab9cbb-1e4b-41b3-b3a1-b324af00b164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:18.948400 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab9cbb-1e4b-41b3-b3a1-b324af00b164 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:18.948678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab9cbb-1e4b-41b3-b3a1-b324af00b164 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:18.948949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab9cbb-1e4b-41b3-b3a1-b324af00b164 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:18.950147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab9cbb-1e4b-41b3-b3a1-b324af00b164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:18.950444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab9cbb-1e4b-41b3-b3a1-b324af00b164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:18.951786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab9cbb-1e4b-41b3-b3a1-b324af00b164 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:18.954214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:18.963334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 544/1093] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:18.967705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:18.969627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab9cbb-1e4b-41b3-b3a1-b324af00b164 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:19.028580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219589bc-b729-43d7-8a6d-6d7762e971ec None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:19.064418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897c7eb-d604-4f69-a7ab-168f0259a1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:19.065957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c7eb-d604-4f69-a7ab-168f0259a1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:19.065957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c7eb-d604-4f69-a7ab-168f0259a1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:19.065957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c7eb-d604-4f69-a7ab-168f0259a1f4 None admin] PATH_INFO: `/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:19.066869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897c7eb-d604-4f69-a7ab-168f0259a1f4 None admin] RBAC: Authorizing `identity:get_project(project_id=7acc94439e334b88b0c9ae0cb8afc6e0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:19.068289 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897c7eb-d604-4f69-a7ab-168f0259a1f4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:19.070003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 550/1094] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:54:18 2022] GET /identity/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:19.076510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897c7eb-d604-4f69-a7ab-168f0259a1f4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:19.079310 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab9cbb-1e4b-41b3-b3a1-b324af00b164 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:19.115412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf395cc3-8cf4-4169-bd3a-e9f28d3bb06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:19.116052 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf395cc3-8cf4-4169-bd3a-e9f28d3bb06b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:19.116339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf395cc3-8cf4-4169-bd3a-e9f28d3bb06b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:19.116616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf395cc3-8cf4-4169-bd3a-e9f28d3bb06b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:19.124703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf395cc3-8cf4-4169-bd3a-e9f28d3bb06b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:19.126178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf395cc3-8cf4-4169-bd3a-e9f28d3bb06b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-bf395cc3-8cf4-4169-bd3a-e9f28d3bb06b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:54:19.137777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:19.138857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:19.140209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:54:19.140209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:54:19.140209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:54:19.140209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:54:19.140209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:54:19.140209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:54:19.140209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:54:19.140209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:19.140209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 545/1095] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:54:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:54:19.144576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf395cc3-8cf4-4169-bd3a-e9f28d3bb06b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:19.180391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897c7eb-d604-4f69-a7ab-168f0259a1f4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:19.194146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:19.201812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:19.214041 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee558-d5bf-4459-9846-90eecf4dc0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:19.214703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee558-d5bf-4459-9846-90eecf4dc0b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:19.215005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee558-d5bf-4459-9846-90eecf4dc0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:19.215287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee558-d5bf-4459-9846-90eecf4dc0b2 None admin] PATH_INFO: `/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0/users/9346bf9dcb3c4433861b9f912a811807/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:19.217367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ee558-d5bf-4459-9846-90eecf4dc0b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7acc94439e334b88b0c9ae0cb8afc6e0, user_id=9346bf9dcb3c4433861b9f912a811807, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:19.218860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ee558-d5bf-4459-9846-90eecf4dc0b2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:19.243374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf395cc3-8cf4-4169-bd3a-e9f28d3bb06b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:19.277153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71109ff5-e8c8-4208-b331-a4846c37bfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:19.277794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71109ff5-e8c8-4208-b331-a4846c37bfba None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:19.278078 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71109ff5-e8c8-4208-b331-a4846c37bfba None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:19.278354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71109ff5-e8c8-4208-b331-a4846c37bfba None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:19.279152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71109ff5-e8c8-4208-b331-a4846c37bfba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:19.279439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71109ff5-e8c8-4208-b331-a4846c37bfba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:19.280848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71109ff5-e8c8-4208-b331-a4846c37bfba None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:19.291378 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 551/1096] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:54:19 2022] PUT /identity/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0/users/9346bf9dcb3c4433861b9f912a811807/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:54:19.292718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 546/1097] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:54:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:19.298777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee558-d5bf-4459-9846-90eecf4dc0b2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:19.408471 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee558-d5bf-4459-9846-90eecf4dc0b2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:19.449168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3e0f0-4410-4878-94d9-a73be338a792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:19.450370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3e0f0-4410-4878-94d9-a73be338a792 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:19.450655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3e0f0-4410-4878-94d9-a73be338a792 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:19.450934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3e0f0-4410-4878-94d9-a73be338a792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:19.451737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3e0f0-4410-4878-94d9-a73be338a792 None admin] RBAC: Adding query filter params (role.id=8e5389a3e81c4e58b716a69b7f74257b, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=7d7bcb4e7c434622b650cdfc500a9680) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:19.452056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3e0f0-4410-4878-94d9-a73be338a792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:19.453511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3e0f0-4410-4878-94d9-a73be338a792 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:19.459522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 552/1098] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:54:19 2022] GET /identity/v3/role_assignments?user.id=7d7bcb4e7c434622b650cdfc500a9680&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=8e5389a3e81c4e58b716a69b7f74257b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:21.326486 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88ec8a-aee9-4057-832d-02e290bfe592 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:21.327017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88ec8a-aee9-4057-832d-02e290bfe592 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:21.327017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88ec8a-aee9-4057-832d-02e290bfe592 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:21.328439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 547/1099] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:21.333797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e340d0-c7c8-459b-9ba8-3c4145bcd4f3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:21.334217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e340d0-c7c8-459b-9ba8-3c4145bcd4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:21.334365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e340d0-c7c8-459b-9ba8-3c4145bcd4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:21.352630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-59e340d0-c7c8-459b-9ba8-3c4145bcd4f3 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:21.353854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-59e340d0-c7c8-459b-9ba8-3c4145bcd4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:21.382507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:21.420996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 553/1100] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:21.442453 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88ec8a-aee9-4057-832d-02e290bfe592 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:21.540847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88ec8a-aee9-4057-832d-02e290bfe592 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:21.556422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:21.573850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12858086-1122-4177-a64d-e513204d44dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:21.575086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858086-1122-4177-a64d-e513204d44dc None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:21.575368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858086-1122-4177-a64d-e513204d44dc None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:21.575684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12858086-1122-4177-a64d-e513204d44dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:21.585847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12858086-1122-4177-a64d-e513204d44dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-12858086-1122-4177-a64d-e513204d44dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:54:21.591945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:21.593777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:21.595517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 548/1101] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:54:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:21.599106 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e340d0-c7c8-459b-9ba8-3c4145bcd4f3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:21.695575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e340d0-c7c8-459b-9ba8-3c4145bcd4f3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:21.727882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0107545-faca-43f6-8605-1750a7d1bf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:21.728492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0107545-faca-43f6-8605-1750a7d1bf60 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:21.728770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0107545-faca-43f6-8605-1750a7d1bf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:21.729067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0107545-faca-43f6-8605-1750a7d1bf60 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:21.729779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0107545-faca-43f6-8605-1750a7d1bf60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:21.730083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0107545-faca-43f6-8605-1750a7d1bf60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:21.731460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0107545-faca-43f6-8605-1750a7d1bf60 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:21.739187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 554/1102] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:54:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:21.745039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12858086-1122-4177-a64d-e513204d44dc None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:21.845250 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12858086-1122-4177-a64d-e513204d44dc None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:21.878875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da30ba-9a62-44f4-82be-d2ca784e1864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:21.879512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da30ba-9a62-44f4-82be-d2ca784e1864 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:21.879799 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da30ba-9a62-44f4-82be-d2ca784e1864 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:21.880079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da30ba-9a62-44f4-82be-d2ca784e1864 None admin] PATH_INFO: `/v3/users/9346bf9dcb3c4433861b9f912a811807` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:21.881272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da30ba-9a62-44f4-82be-d2ca784e1864 None admin] RBAC: Authorizing `identity:get_user(user_id=9346bf9dcb3c4433861b9f912a811807)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:21.882711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da30ba-9a62-44f4-82be-d2ca784e1864 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:21.884316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 549/1103] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:21 2022] GET /identity/v3/users/9346bf9dcb3c4433861b9f912a811807 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:21.890110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0107545-faca-43f6-8605-1750a7d1bf60 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:21.920313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:21.986922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0107545-faca-43f6-8605-1750a7d1bf60 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:22.019279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb9977-d0c5-48de-915d-75e0198f8bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:22.019886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb9977-d0c5-48de-915d-75e0198f8bfd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:22.020170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb9977-d0c5-48de-915d-75e0198f8bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:22.020444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb9977-d0c5-48de-915d-75e0198f8bfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:22.084855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb9977-d0c5-48de-915d-75e0198f8bfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:22.086324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb9977-d0c5-48de-915d-75e0198f8bfd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:22.114168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:22.182262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 555/1104] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:54:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:54:22.188663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da30ba-9a62-44f4-82be-d2ca784e1864 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:22.233290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:22.286292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da30ba-9a62-44f4-82be-d2ca784e1864 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:22.318720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ac8ff-6c94-4dc2-a923-c9b78b8b649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:22.319347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ac8ff-6c94-4dc2-a923-c9b78b8b649d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:22.319632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ac8ff-6c94-4dc2-a923-c9b78b8b649d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:22.319908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ac8ff-6c94-4dc2-a923-c9b78b8b649d None admin] PATH_INFO: `/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:22.321089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ac8ff-6c94-4dc2-a923-c9b78b8b649d None admin] RBAC: Authorizing `identity:get_project(project_id=7acc94439e334b88b0c9ae0cb8afc6e0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:22.322520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ac8ff-6c94-4dc2-a923-c9b78b8b649d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:22.324155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 550/1105] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:54:22 2022] GET /identity/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:22.330697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb9977-d0c5-48de-915d-75e0198f8bfd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:22.428635 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb9977-d0c5-48de-915d-75e0198f8bfd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:22.457131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:22.462746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e11c23-03be-47d0-a025-dca57c06619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:22.463355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e11c23-03be-47d0-a025-dca57c06619c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:22.463637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e11c23-03be-47d0-a025-dca57c06619c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:22.463908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e11c23-03be-47d0-a025-dca57c06619c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:22.464717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e11c23-03be-47d0-a025-dca57c06619c None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=7acc94439e334b88b0c9ae0cb8afc6e0, user.id=9346bf9dcb3c4433861b9f912a811807) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:22.465059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e11c23-03be-47d0-a025-dca57c06619c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:22.469879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e11c23-03be-47d0-a025-dca57c06619c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:22.472049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 556/1106] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:54:22 2022] GET /identity/v3/role_assignments?user.id=9346bf9dcb3c4433861b9f912a811807&scope.project.id=7acc94439e334b88b0c9ae0cb8afc6e0&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:24.470187 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0f4a-e73f-4f8e-b70f-832883eff4c7 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:24.470655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0f4a-e73f-4f8e-b70f-832883eff4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:24.470761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0f4a-e73f-4f8e-b70f-832883eff4c7 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:24.472114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 551/1107] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:24.477179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61a6d-657e-4479-9f3c-4d0cb00e88a2 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:24.477565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61a6d-657e-4479-9f3c-4d0cb00e88a2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:24.477738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61a6d-657e-4479-9f3c-4d0cb00e88a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:24.496014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b1e61a6d-657e-4479-9f3c-4d0cb00e88a2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:24.497269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b1e61a6d-657e-4479-9f3c-4d0cb00e88a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:24.562036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 557/1108] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:24.585766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff0f4a-e73f-4f8e-b70f-832883eff4c7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:24.686878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff0f4a-e73f-4f8e-b70f-832883eff4c7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:24.699725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:24.728727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68355853-a527-4b49-88e9-8159549f0c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:24.729467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68355853-a527-4b49-88e9-8159549f0c0f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:24.729743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68355853-a527-4b49-88e9-8159549f0c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:24.730017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68355853-a527-4b49-88e9-8159549f0c0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:24.731802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68355853-a527-4b49-88e9-8159549f0c0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:24.733298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68355853-a527-4b49-88e9-8159549f0c0f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:24.734940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 552/1109] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:24.742555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e61a6d-657e-4479-9f3c-4d0cb00e88a2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:24.838256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e61a6d-657e-4479-9f3c-4d0cb00e88a2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:24.861017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:24.871441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4d60d-29b2-4598-90c9-0a61103234f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:24.872053 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4d60d-29b2-4598-90c9-0a61103234f1 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:24.872339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4d60d-29b2-4598-90c9-0a61103234f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:24.872616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4d60d-29b2-4598-90c9-0a61103234f1 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:24.873344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4d60d-29b2-4598-90c9-0a61103234f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:24.875023 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4d60d-29b2-4598-90c9-0a61103234f1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:24.962663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 558/1110] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:54:24 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:27.029439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefeb34-d5b1-4dfe-950b-35beff7a0a19 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:27.029971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefeb34-d5b1-4dfe-950b-35beff7a0a19 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:27.029971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefeb34-d5b1-4dfe-950b-35beff7a0a19 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:27.031336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 553/1111] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:27.036408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe37e4-6b8b-43f7-97d8-72865c715375 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:27.036804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe37e4-6b8b-43f7-97d8-72865c715375 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:27.036955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe37e4-6b8b-43f7-97d8-72865c715375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:27.055250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-adfe37e4-6b8b-43f7-97d8-72865c715375 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:27.056425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-adfe37e4-6b8b-43f7-97d8-72865c715375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:27.126685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 559/1112] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:27.149514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefeb34-d5b1-4dfe-950b-35beff7a0a19 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:27.249349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefeb34-d5b1-4dfe-950b-35beff7a0a19 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:27.280842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede51389-791b-4f16-8715-c11fc7a4ecb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:27.281432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede51389-791b-4f16-8715-c11fc7a4ecb8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:27.281712 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede51389-791b-4f16-8715-c11fc7a4ecb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:27.281984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede51389-791b-4f16-8715-c11fc7a4ecb8 None admin] PATH_INFO: `/v3/roles/f4c7e5a278e34f03a86f1f88ed77db0c` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:27.283415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede51389-791b-4f16-8715-c11fc7a4ecb8 None admin] RBAC: Authorizing `identity:get_role(role_id=f4c7e5a278e34f03a86f1f88ed77db0c)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:27.284765 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede51389-791b-4f16-8715-c11fc7a4ecb8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:27.286112 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 554/1113] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:27 2022] GET /identity/v3/roles/f4c7e5a278e34f03a86f1f88ed77db0c => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:27.292077 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe37e4-6b8b-43f7-97d8-72865c715375 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:27.390029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe37e4-6b8b-43f7-97d8-72865c715375 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:27.400194 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:27.423658 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a9cf4-b743-4261-8ffa-ce8e01cd11fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:27.424306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9cf4-b743-4261-8ffa-ce8e01cd11fb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:27.425549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9cf4-b743-4261-8ffa-ce8e01cd11fb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:27.425549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9cf4-b743-4261-8ffa-ce8e01cd11fb None admin] PATH_INFO: `/v3/users/e8315b1c0fc94471ae6ff64bcb2175a1` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:27.437897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a9cf4-b743-4261-8ffa-ce8e01cd11fb None admin] RBAC: Authorizing `identity:get_user(user_id=e8315b1c0fc94471ae6ff64bcb2175a1)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:27.439338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a9cf4-b743-4261-8ffa-ce8e01cd11fb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:27.441025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 560/1114] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:27 2022] GET /identity/v3/users/e8315b1c0fc94471ae6ff64bcb2175a1 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:27.446791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede51389-791b-4f16-8715-c11fc7a4ecb8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:27.498147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:27.547920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede51389-791b-4f16-8715-c11fc7a4ecb8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:27.579579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65141dd8-1f62-478d-9269-95df852a18f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:27.580186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65141dd8-1f62-478d-9269-95df852a18f6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:27.580465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65141dd8-1f62-478d-9269-95df852a18f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:27.580736 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65141dd8-1f62-478d-9269-95df852a18f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:27.645595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65141dd8-1f62-478d-9269-95df852a18f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:27.647027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65141dd8-1f62-478d-9269-95df852a18f6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:27.692853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:27.742773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 555/1115] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:54:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:54:27.749203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a9cf4-b743-4261-8ffa-ce8e01cd11fb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:27.849856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a9cf4-b743-4261-8ffa-ce8e01cd11fb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:27.882875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f728e7e-4a05-41d0-be7c-8d83457a3519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:27.883506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728e7e-4a05-41d0-be7c-8d83457a3519 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:27.883788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728e7e-4a05-41d0-be7c-8d83457a3519 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:27.884062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728e7e-4a05-41d0-be7c-8d83457a3519 None admin] PATH_INFO: `/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:27.885241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f728e7e-4a05-41d0-be7c-8d83457a3519 None admin] RBAC: Authorizing `identity:get_project(project_id=7acc94439e334b88b0c9ae0cb8afc6e0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:27.887163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f728e7e-4a05-41d0-be7c-8d83457a3519 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:27.888814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 561/1116] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:54:27 2022] GET /identity/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:27.895013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65141dd8-1f62-478d-9269-95df852a18f6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:27.989725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65141dd8-1f62-478d-9269-95df852a18f6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:28.022670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5baf76e-58c3-4e3a-9d67-174be175d9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:28.023265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baf76e-58c3-4e3a-9d67-174be175d9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:28.023545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baf76e-58c3-4e3a-9d67-174be175d9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:28.023818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5baf76e-58c3-4e3a-9d67-174be175d9a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:28.024594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5baf76e-58c3-4e3a-9d67-174be175d9a8 None admin] RBAC: Adding query filter params (role.id=f4c7e5a278e34f03a86f1f88ed77db0c, scope.project.id=7acc94439e334b88b0c9ae0cb8afc6e0, user.id=e8315b1c0fc94471ae6ff64bcb2175a1) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:28.024900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5baf76e-58c3-4e3a-9d67-174be175d9a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:28.026340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5baf76e-58c3-4e3a-9d67-174be175d9a8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:28.032114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 556/1117] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:54:27 2022] GET /identity/v3/role_assignments?user.id=e8315b1c0fc94471ae6ff64bcb2175a1&scope.project.id=7acc94439e334b88b0c9ae0cb8afc6e0&role.id=f4c7e5a278e34f03a86f1f88ed77db0c => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:30.015438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17386d-87f5-41a6-84ec-88a49b4c6a4b None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.015975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17386d-87f5-41a6-84ec-88a49b4c6a4b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.015975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17386d-87f5-41a6-84ec-88a49b4c6a4b None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.017391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 562/1118] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:30.022659 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30fef8-e27c-4baa-a5d1-374f5b9aad0c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.023077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30fef8-e27c-4baa-a5d1-374f5b9aad0c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.023214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30fef8-e27c-4baa-a5d1-374f5b9aad0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.040866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6c30fef8-e27c-4baa-a5d1-374f5b9aad0c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:30.042066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6c30fef8-e27c-4baa-a5d1-374f5b9aad0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:30.049567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:30.107322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 557/1119] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:30.129149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17386d-87f5-41a6-84ec-88a49b4c6a4b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:30.164697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:30.227876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17386d-87f5-41a6-84ec-88a49b4c6a4b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:30.260797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70a4b6-e511-4092-be3b-7b66342b5b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:30.261948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70a4b6-e511-4092-be3b-7b66342b5b14 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.262229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70a4b6-e511-4092-be3b-7b66342b5b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.262498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70a4b6-e511-4092-be3b-7b66342b5b14 None admin] PATH_INFO: `/v3/roles/f4c7e5a278e34f03a86f1f88ed77db0c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.264008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70a4b6-e511-4092-be3b-7b66342b5b14 None admin] RBAC: Authorizing `identity:get_role(role_id=f4c7e5a278e34f03a86f1f88ed77db0c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:30.265394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70a4b6-e511-4092-be3b-7b66342b5b14 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:30.266797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 563/1120] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:30 2022] GET /identity/v3/roles/f4c7e5a278e34f03a86f1f88ed77db0c => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:30.272721 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30fef8-e27c-4baa-a5d1-374f5b9aad0c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:30.367401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30fef8-e27c-4baa-a5d1-374f5b9aad0c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:30.399983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac8622-32ce-4532-a967-fe8b17a86ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:30.400604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac8622-32ce-4532-a967-fe8b17a86ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.400882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac8622-32ce-4532-a967-fe8b17a86ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.401176 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac8622-32ce-4532-a967-fe8b17a86ac3 None admin] PATH_INFO: `/v3/users/e8315b1c0fc94471ae6ff64bcb2175a1` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.402940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac8622-32ce-4532-a967-fe8b17a86ac3 None admin] RBAC: Authorizing `identity:get_user(user_id=e8315b1c0fc94471ae6ff64bcb2175a1)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:30.404355 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac8622-32ce-4532-a967-fe8b17a86ac3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:30.405955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 558/1121] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:30 2022] GET /identity/v3/users/e8315b1c0fc94471ae6ff64bcb2175a1 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:30.411491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70a4b6-e511-4092-be3b-7b66342b5b14 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:30.508122 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70a4b6-e511-4092-be3b-7b66342b5b14 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:30.510371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:30.552555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7e12a-cea5-481c-a55e-b2290cde851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:30.553157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7e12a-cea5-481c-a55e-b2290cde851d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.553439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7e12a-cea5-481c-a55e-b2290cde851d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.553715 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7e12a-cea5-481c-a55e-b2290cde851d None admin] PATH_INFO: `/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.555222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7e12a-cea5-481c-a55e-b2290cde851d None admin] RBAC: Authorizing `identity:get_project(project_id=7acc94439e334b88b0c9ae0cb8afc6e0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:30.556690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7e12a-cea5-481c-a55e-b2290cde851d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:30.559414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 564/1122] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:54:30 2022] GET /identity/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:30.565166 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac8622-32ce-4532-a967-fe8b17a86ac3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:30.577870 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:30.662761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac8622-32ce-4532-a967-fe8b17a86ac3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:30.695528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b9616-df50-4978-a2b6-e5a8d4ba7383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:30.696178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9616-df50-4978-a2b6-e5a8d4ba7383 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.696463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9616-df50-4978-a2b6-e5a8d4ba7383 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.696740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9616-df50-4978-a2b6-e5a8d4ba7383 None admin] PATH_INFO: `/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0/users/e8315b1c0fc94471ae6ff64bcb2175a1/roles/f4c7e5a278e34f03a86f1f88ed77db0c` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.699507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b9616-df50-4978-a2b6-e5a8d4ba7383 None admin] RBAC: Authorizing `identity:create_grant(project_id=7acc94439e334b88b0c9ae0cb8afc6e0, user_id=e8315b1c0fc94471ae6ff64bcb2175a1, role_id=f4c7e5a278e34f03a86f1f88ed77db0c)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:30.700943 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b9616-df50-4978-a2b6-e5a8d4ba7383 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:30.790752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 559/1123] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:54:30 2022] PUT /identity/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0/users/e8315b1c0fc94471ae6ff64bcb2175a1/roles/f4c7e5a278e34f03a86f1f88ed77db0c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:32.767192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21657fb-5719-45b3-8975-74c7a4f0317d None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:32.767691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21657fb-5719-45b3-8975-74c7a4f0317d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:32.767691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21657fb-5719-45b3-8975-74c7a4f0317d None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:32.769146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 565/1124] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:32.774336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d1cda-df59-4f42-a69d-49eb20f9c5dd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:32.774751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d1cda-df59-4f42-a69d-49eb20f9c5dd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:32.774896 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d1cda-df59-4f42-a69d-49eb20f9c5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:32.793683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7f8d1cda-df59-4f42-a69d-49eb20f9c5dd None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:32.794876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7f8d1cda-df59-4f42-a69d-49eb20f9c5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:32.854488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:32.859481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 560/1125] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:32.882388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21657fb-5719-45b3-8975-74c7a4f0317d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:32.980804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21657fb-5719-45b3-8975-74c7a4f0317d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:33.013882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c8f8b-c661-4596-a829-4542db7b32d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:33.014504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c8f8b-c661-4596-a829-4542db7b32d0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:33.014783 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c8f8b-c661-4596-a829-4542db7b32d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:33.015055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c8f8b-c661-4596-a829-4542db7b32d0 None admin] PATH_INFO: `/v3/roles/f4c7e5a278e34f03a86f1f88ed77db0c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:33.016483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c8f8b-c661-4596-a829-4542db7b32d0 None admin] RBAC: Authorizing `identity:get_role(role_id=f4c7e5a278e34f03a86f1f88ed77db0c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:33.017912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c8f8b-c661-4596-a829-4542db7b32d0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:33.019178 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 566/1126] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:32 2022] GET /identity/v3/roles/f4c7e5a278e34f03a86f1f88ed77db0c => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:33.024890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d1cda-df59-4f42-a69d-49eb20f9c5dd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:33.120799 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d1cda-df59-4f42-a69d-49eb20f9c5dd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:33.153931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a46d4-4eb8-4333-ae18-750fe3afae9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:33.154561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a46d4-4eb8-4333-ae18-750fe3afae9c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:33.154857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a46d4-4eb8-4333-ae18-750fe3afae9c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:33.155137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a46d4-4eb8-4333-ae18-750fe3afae9c None admin] PATH_INFO: `/v3/users/e8315b1c0fc94471ae6ff64bcb2175a1` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:33.156320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a46d4-4eb8-4333-ae18-750fe3afae9c None admin] RBAC: Authorizing `identity:get_user(user_id=e8315b1c0fc94471ae6ff64bcb2175a1)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:33.157777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a46d4-4eb8-4333-ae18-750fe3afae9c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:33.159824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 561/1127] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:33 2022] GET /identity/v3/users/e8315b1c0fc94471ae6ff64bcb2175a1 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:33.165549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c8f8b-c661-4596-a829-4542db7b32d0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:33.207273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:33.263605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c8f8b-c661-4596-a829-4542db7b32d0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:33.296424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10edcf6-d120-4d25-a785-7bc6a90f02c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:33.296997 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10edcf6-d120-4d25-a785-7bc6a90f02c8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:33.297287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10edcf6-d120-4d25-a785-7bc6a90f02c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:33.297567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10edcf6-d120-4d25-a785-7bc6a90f02c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:33.364196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10edcf6-d120-4d25-a785-7bc6a90f02c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:33.365669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10edcf6-d120-4d25-a785-7bc6a90f02c8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:33.404576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:33.461153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 567/1128] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:54:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:54:33.467749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a46d4-4eb8-4333-ae18-750fe3afae9c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:33.532564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:33.565430 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a46d4-4eb8-4333-ae18-750fe3afae9c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:33.598465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a32de1-02c7-4f3d-b237-6c062a256def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:33.599087 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a32de1-02c7-4f3d-b237-6c062a256def None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:33.599369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a32de1-02c7-4f3d-b237-6c062a256def None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:33.599644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a32de1-02c7-4f3d-b237-6c062a256def None admin] PATH_INFO: `/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:33.600787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a32de1-02c7-4f3d-b237-6c062a256def None admin] RBAC: Authorizing `identity:get_project(project_id=7acc94439e334b88b0c9ae0cb8afc6e0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:33.602245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a32de1-02c7-4f3d-b237-6c062a256def None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:33.603921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 562/1129] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:54:33 2022] GET /identity/v3/projects/7acc94439e334b88b0c9ae0cb8afc6e0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:33.610669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10edcf6-d120-4d25-a785-7bc6a90f02c8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:33.705694 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10edcf6-d120-4d25-a785-7bc6a90f02c8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:33.738815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720dba9-c1f3-454f-ad04-8edb90dad459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:33.739415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720dba9-c1f3-454f-ad04-8edb90dad459 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:33.739692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720dba9-c1f3-454f-ad04-8edb90dad459 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:33.739965 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720dba9-c1f3-454f-ad04-8edb90dad459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:33.740745 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720dba9-c1f3-454f-ad04-8edb90dad459 None admin] RBAC: Adding query filter params (role.id=f4c7e5a278e34f03a86f1f88ed77db0c, scope.project.id=7acc94439e334b88b0c9ae0cb8afc6e0, user.id=e8315b1c0fc94471ae6ff64bcb2175a1) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:33.741149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720dba9-c1f3-454f-ad04-8edb90dad459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:33.742564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720dba9-c1f3-454f-ad04-8edb90dad459 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:33.748178 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 568/1130] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:54:33 2022] GET /identity/v3/role_assignments?user.id=e8315b1c0fc94471ae6ff64bcb2175a1&scope.project.id=7acc94439e334b88b0c9ae0cb8afc6e0&role.id=f4c7e5a278e34f03a86f1f88ed77db0c => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:35.741027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2b976-ae1b-4bdc-9e5b-7f33c83465a6 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.741532 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2b976-ae1b-4bdc-9e5b-7f33c83465a6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.741532 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2b976-ae1b-4bdc-9e5b-7f33c83465a6 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.742879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 563/1131] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:35.748803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686c565-45d9-4b41-8116-fa0400ec07a1 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.749271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686c565-45d9-4b41-8116-fa0400ec07a1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.749395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686c565-45d9-4b41-8116-fa0400ec07a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.756728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:35.767604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0686c565-45d9-4b41-8116-fa0400ec07a1 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:35.768804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0686c565-45d9-4b41-8116-fa0400ec07a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:35.832057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 569/1132] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:35.854456 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2b976-ae1b-4bdc-9e5b-7f33c83465a6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:35.953350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2b976-ae1b-4bdc-9e5b-7f33c83465a6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:35.985584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c38a37-6f90-4597-b72d-c29c2f3dea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:35.986214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38a37-6f90-4597-b72d-c29c2f3dea32 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.986500 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38a37-6f90-4597-b72d-c29c2f3dea32 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.986780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38a37-6f90-4597-b72d-c29c2f3dea32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.987917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c38a37-6f90-4597-b72d-c29c2f3dea32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:35.989387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c38a37-6f90-4597-b72d-c29c2f3dea32 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:35.990959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 564/1133] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:35.997140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686c565-45d9-4b41-8116-fa0400ec07a1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:36.092780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686c565-45d9-4b41-8116-fa0400ec07a1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:36.126304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92af1a6-abd1-4643-b19d-13079aa0ac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:36.126919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92af1a6-abd1-4643-b19d-13079aa0ac88 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:36.127202 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92af1a6-abd1-4643-b19d-13079aa0ac88 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:36.127479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92af1a6-abd1-4643-b19d-13079aa0ac88 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:36.128170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92af1a6-abd1-4643-b19d-13079aa0ac88 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:36.130548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92af1a6-abd1-4643-b19d-13079aa0ac88 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:36.134235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:36.199751 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 570/1134] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:54:35 2022] POST /identity/v3/projects => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:38.277862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c43f1b-1e7e-4792-9abc-4e6fe07c6a8f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:38.278504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c43f1b-1e7e-4792-9abc-4e6fe07c6a8f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:38.278504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c43f1b-1e7e-4792-9abc-4e6fe07c6a8f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:38.279786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 565/1135] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:38.284916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7583-76a0-4c78-8cb3-46f977aad130 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:38.285361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7583-76a0-4c78-8cb3-46f977aad130 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:38.285506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7583-76a0-4c78-8cb3-46f977aad130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:38.303660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c42e7583-76a0-4c78-8cb3-46f977aad130 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:38.304850 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c42e7583-76a0-4c78-8cb3-46f977aad130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:38.368906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 571/1136] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:38.393958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c43f1b-1e7e-4792-9abc-4e6fe07c6a8f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:38.397770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:38.494117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c43f1b-1e7e-4792-9abc-4e6fe07c6a8f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:38.527287 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff80b1e-4757-4342-9668-57077a4d3ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:38.527970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff80b1e-4757-4342-9668-57077a4d3ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:38.528203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff80b1e-4757-4342-9668-57077a4d3ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:38.528478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff80b1e-4757-4342-9668-57077a4d3ad7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:38.529651 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff80b1e-4757-4342-9668-57077a4d3ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:38.531080 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff80b1e-4757-4342-9668-57077a4d3ad7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:38.532676 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 566/1137] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:54:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:38.538417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e7583-76a0-4c78-8cb3-46f977aad130 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:38.636434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e7583-76a0-4c78-8cb3-46f977aad130 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:38.652521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:38.670054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ca389-9c1e-4534-970c-57c9bbdb3e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:38.670674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ca389-9c1e-4534-970c-57c9bbdb3e27 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:38.671030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ca389-9c1e-4534-970c-57c9bbdb3e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:38.671307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ca389-9c1e-4534-970c-57c9bbdb3e27 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:38.671989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ca389-9c1e-4534-970c-57c9bbdb3e27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:38.673455 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ca389-9c1e-4534-970c-57c9bbdb3e27 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:38.764705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 572/1138] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:54:38 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:40.791910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdacbf7-8cac-45a6-a524-050b98ae30ed None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:40.792928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdacbf7-8cac-45a6-a524-050b98ae30ed None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:40.792928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdacbf7-8cac-45a6-a524-050b98ae30ed None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:40.794308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 567/1139] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:40.799494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c42c8f-7fe7-4957-9abe-51c21f7af0d5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:40.799888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c42c8f-7fe7-4957-9abe-51c21f7af0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:40.800056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c42c8f-7fe7-4957-9abe-51c21f7af0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:40.818280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-10c42c8f-7fe7-4957-9abe-51c21f7af0d5 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:40.819469 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-10c42c8f-7fe7-4957-9abe-51c21f7af0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:40.884845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 573/1140] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:40.910218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdacbf7-8cac-45a6-a524-050b98ae30ed None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:40.985497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:41.014013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdacbf7-8cac-45a6-a524-050b98ae30ed None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:41.047809 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077148e-d654-4a93-b6b7-e958f48679ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:41.048385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077148e-d654-4a93-b6b7-e958f48679ee None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:41.048670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077148e-d654-4a93-b6b7-e958f48679ee None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:41.048947 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077148e-d654-4a93-b6b7-e958f48679ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:41.058823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077148e-d654-4a93-b6b7-e958f48679ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b077148e-d654-4a93-b6b7-e958f48679ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:54:41.063680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:41.064924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:41.068582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 568/1141] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:54:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:41.070998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c42c8f-7fe7-4957-9abe-51c21f7af0d5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:41.165815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c42c8f-7fe7-4957-9abe-51c21f7af0d5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:41.167806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:41.201576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cc5b6-f1d6-41cf-8a11-f27c9710b20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:41.202196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc5b6-f1d6-41cf-8a11-f27c9710b20e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:41.202483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc5b6-f1d6-41cf-8a11-f27c9710b20e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:41.202762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc5b6-f1d6-41cf-8a11-f27c9710b20e None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:41.203493 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cc5b6-f1d6-41cf-8a11-f27c9710b20e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:41.203803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cc5b6-f1d6-41cf-8a11-f27c9710b20e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:41.205204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cc5b6-f1d6-41cf-8a11-f27c9710b20e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:41.213106 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 574/1142] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:54:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:41.219545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077148e-d654-4a93-b6b7-e958f48679ee None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:41.316009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077148e-d654-4a93-b6b7-e958f48679ee None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:41.346951 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:41.351718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a37766-3feb-4698-80fb-6c0be9f120c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:41.354934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a37766-3feb-4698-80fb-6c0be9f120c5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:41.354934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a37766-3feb-4698-80fb-6c0be9f120c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:41.354934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a37766-3feb-4698-80fb-6c0be9f120c5 None admin] PATH_INFO: `/v3/users/56357eda81124e6483a36803c6030ddf` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:41.365451 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a37766-3feb-4698-80fb-6c0be9f120c5 None admin] RBAC: Authorizing `identity:get_user(user_id=56357eda81124e6483a36803c6030ddf)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:41.366907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a37766-3feb-4698-80fb-6c0be9f120c5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:41.368507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 569/1143] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:41 2022] GET /identity/v3/users/56357eda81124e6483a36803c6030ddf => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:41.374640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cc5b6-f1d6-41cf-8a11-f27c9710b20e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:41.473585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cc5b6-f1d6-41cf-8a11-f27c9710b20e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:41.505796 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45883a97-567e-42ec-b211-fe01067d9112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:41.506416 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45883a97-567e-42ec-b211-fe01067d9112 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:41.506698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45883a97-567e-42ec-b211-fe01067d9112 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:41.506977 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45883a97-567e-42ec-b211-fe01067d9112 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:41.536887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:41.572123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45883a97-567e-42ec-b211-fe01067d9112 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:41.574065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45883a97-567e-42ec-b211-fe01067d9112 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:41.668140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 575/1144] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:54:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:54:41.675074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a37766-3feb-4698-80fb-6c0be9f120c5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:41.772916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a37766-3feb-4698-80fb-6c0be9f120c5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:41.805990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0898ff7-cc9b-4703-ae13-9bf5c4f968d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:41.806627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0898ff7-cc9b-4703-ae13-9bf5c4f968d8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:41.806913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0898ff7-cc9b-4703-ae13-9bf5c4f968d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:41.807199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0898ff7-cc9b-4703-ae13-9bf5c4f968d8 None admin] PATH_INFO: `/v3/projects/93084177668949398520cd51f07d74dc` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:41.808888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0898ff7-cc9b-4703-ae13-9bf5c4f968d8 None admin] RBAC: Authorizing `identity:get_project(project_id=93084177668949398520cd51f07d74dc)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:41.810340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0898ff7-cc9b-4703-ae13-9bf5c4f968d8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:41.812003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 570/1145] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:54:41 2022] GET /identity/v3/projects/93084177668949398520cd51f07d74dc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:41.818018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45883a97-567e-42ec-b211-fe01067d9112 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:41.878048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:41.914270 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45883a97-567e-42ec-b211-fe01067d9112 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:41.946892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0a498-8658-47bb-818b-423c258869ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:41.947501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0a498-8658-47bb-818b-423c258869ff None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:41.947788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0a498-8658-47bb-818b-423c258869ff None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:41.948068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0a498-8658-47bb-818b-423c258869ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:41.949496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0a498-8658-47bb-818b-423c258869ff None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=93084177668949398520cd51f07d74dc, user.id=56357eda81124e6483a36803c6030ddf) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:41.949810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0a498-8658-47bb-818b-423c258869ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:41.951230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0a498-8658-47bb-818b-423c258869ff None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:41.956477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 576/1146] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:54:41 2022] GET /identity/v3/role_assignments?user.id=56357eda81124e6483a36803c6030ddf&scope.project.id=93084177668949398520cd51f07d74dc&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:54:43.976244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024a7e9-bdd0-4e6e-ab7b-f693c1a9d284 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:43.976834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024a7e9-bdd0-4e6e-ab7b-f693c1a9d284 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:43.976834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024a7e9-bdd0-4e6e-ab7b-f693c1a9d284 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:43.978197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 571/1147] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:43.985074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda95804-b7ce-49a2-ab1f-65c5ca2a0a7b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:43.985512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda95804-b7ce-49a2-ab1f-65c5ca2a0a7b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:43.985929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda95804-b7ce-49a2-ab1f-65c5ca2a0a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:44.012093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-fda95804-b7ce-49a2-ab1f-65c5ca2a0a7b None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:44.013810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fda95804-b7ce-49a2-ab1f-65c5ca2a0a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:44.085931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 577/1148] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:44.108912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024a7e9-bdd0-4e6e-ab7b-f693c1a9d284 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:44.166075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:44.206293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024a7e9-bdd0-4e6e-ab7b-f693c1a9d284 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:44.240316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b4ad6-fdd3-453f-9902-20fa967687b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:44.240884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b4ad6-fdd3-453f-9902-20fa967687b4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:44.241192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b4ad6-fdd3-453f-9902-20fa967687b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:44.241466 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b4ad6-fdd3-453f-9902-20fa967687b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:44.251589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b4ad6-fdd3-453f-9902-20fa967687b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-cb3b4ad6-fdd3-453f-9902-20fa967687b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:54:44.256448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:44.257512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:44.258764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 572/1149] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:54:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:44.263559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda95804-b7ce-49a2-ab1f-65c5ca2a0a7b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:44.320585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:44.361477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda95804-b7ce-49a2-ab1f-65c5ca2a0a7b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:44.395928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc35cc-7d67-47bb-95b1-50ce814ae34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:44.396539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc35cc-7d67-47bb-95b1-50ce814ae34b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:44.396825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc35cc-7d67-47bb-95b1-50ce814ae34b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:44.397135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc35cc-7d67-47bb-95b1-50ce814ae34b None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:44.397858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc35cc-7d67-47bb-95b1-50ce814ae34b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:44.398164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc35cc-7d67-47bb-95b1-50ce814ae34b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:44.399532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc35cc-7d67-47bb-95b1-50ce814ae34b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:44.407990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 578/1150] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:54:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:44.413845 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b4ad6-fdd3-453f-9902-20fa967687b4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:44.512897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b4ad6-fdd3-453f-9902-20fa967687b4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:44.529593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:44.546160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498dedd9-e7ba-406d-8899-bad5b6fc093f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:44.546896 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dedd9-e7ba-406d-8899-bad5b6fc093f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:44.547271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dedd9-e7ba-406d-8899-bad5b6fc093f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:44.547649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dedd9-e7ba-406d-8899-bad5b6fc093f None admin] PATH_INFO: `/v3/users/56357eda81124e6483a36803c6030ddf` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:44.548902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498dedd9-e7ba-406d-8899-bad5b6fc093f None admin] RBAC: Authorizing `identity:get_user(user_id=56357eda81124e6483a36803c6030ddf)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:44.550358 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498dedd9-e7ba-406d-8899-bad5b6fc093f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:44.551965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 573/1151] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:44 2022] GET /identity/v3/users/56357eda81124e6483a36803c6030ddf => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:44.558562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc35cc-7d67-47bb-95b1-50ce814ae34b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:44.655260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc35cc-7d67-47bb-95b1-50ce814ae34b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:44.674633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:44.688002 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fdb82-a022-4dbb-bb6d-69dfa796b921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:44.688638 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fdb82-a022-4dbb-bb6d-69dfa796b921 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:44.688921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fdb82-a022-4dbb-bb6d-69dfa796b921 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:44.689228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fdb82-a022-4dbb-bb6d-69dfa796b921 None admin] PATH_INFO: `/v3/projects/93084177668949398520cd51f07d74dc` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:44.690377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583fdb82-a022-4dbb-bb6d-69dfa796b921 None admin] RBAC: Authorizing `identity:get_project(project_id=93084177668949398520cd51f07d74dc)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:44.691806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583fdb82-a022-4dbb-bb6d-69dfa796b921 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:44.693500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 579/1152] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:54:44 2022] GET /identity/v3/projects/93084177668949398520cd51f07d74dc => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:44.699437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498dedd9-e7ba-406d-8899-bad5b6fc093f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:44.796996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498dedd9-e7ba-406d-8899-bad5b6fc093f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:44.829622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ada7e-26b2-4695-8f8a-ee2151818138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:44.830280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ada7e-26b2-4695-8f8a-ee2151818138 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:44.830561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ada7e-26b2-4695-8f8a-ee2151818138 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:44.830835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ada7e-26b2-4695-8f8a-ee2151818138 None admin] PATH_INFO: `/v3/projects/93084177668949398520cd51f07d74dc/users/56357eda81124e6483a36803c6030ddf/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:44.832767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ada7e-26b2-4695-8f8a-ee2151818138 None admin] RBAC: Authorizing `identity:create_grant(project_id=93084177668949398520cd51f07d74dc, user_id=56357eda81124e6483a36803c6030ddf, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:44.834224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ada7e-26b2-4695-8f8a-ee2151818138 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:44.927311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 574/1153] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:54:44 2022] PUT /identity/v3/projects/93084177668949398520cd51f07d74dc/users/56357eda81124e6483a36803c6030ddf/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:46.881676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50204fe-fd1b-4a9d-9c0d-22fa4109b14e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:46.882242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50204fe-fd1b-4a9d-9c0d-22fa4109b14e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:46.882242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50204fe-fd1b-4a9d-9c0d-22fa4109b14e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:46.883583 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 580/1154] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:46.888618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8425ef-4872-4a7c-bab5-48cc4c64afa1 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:46.889063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8425ef-4872-4a7c-bab5-48cc4c64afa1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:46.889239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8425ef-4872-4a7c-bab5-48cc4c64afa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:46.907734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-fe8425ef-4872-4a7c-bab5-48cc4c64afa1 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:46.908932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fe8425ef-4872-4a7c-bab5-48cc4c64afa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:46.949593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:46.973599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 575/1155] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:46.996594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50204fe-fd1b-4a9d-9c0d-22fa4109b14e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:47.094887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50204fe-fd1b-4a9d-9c0d-22fa4109b14e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:47.127315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49722cf-019a-4a8c-a55b-2d0b07b8d0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:47.127939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49722cf-019a-4a8c-a55b-2d0b07b8d0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:47.128225 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49722cf-019a-4a8c-a55b-2d0b07b8d0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:47.128504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49722cf-019a-4a8c-a55b-2d0b07b8d0f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:47.138922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49722cf-019a-4a8c-a55b-2d0b07b8d0f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f49722cf-019a-4a8c-a55b-2d0b07b8d0f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:54:47.143757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:47.144851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:54:47.145879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 581/1156] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:54:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:47.150885 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8425ef-4872-4a7c-bab5-48cc4c64afa1 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:47.249203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8425ef-4872-4a7c-bab5-48cc4c64afa1 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:47.283230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfc082-bf2f-4b17-9098-f059c845efd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:47.283843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfc082-bf2f-4b17-9098-f059c845efd7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:47.284133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfc082-bf2f-4b17-9098-f059c845efd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:47.284409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfc082-bf2f-4b17-9098-f059c845efd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:47.285155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfc082-bf2f-4b17-9098-f059c845efd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:47.285467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfc082-bf2f-4b17-9098-f059c845efd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:47.286830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfc082-bf2f-4b17-9098-f059c845efd7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:47.295322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 576/1157] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:54:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:47.301054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49722cf-019a-4a8c-a55b-2d0b07b8d0f3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:47.349565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:47.414362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49722cf-019a-4a8c-a55b-2d0b07b8d0f3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:47.454934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3514b12-72a8-4192-9266-a4505d70fed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:47.455574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3514b12-72a8-4192-9266-a4505d70fed5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:47.455859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3514b12-72a8-4192-9266-a4505d70fed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:47.456137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3514b12-72a8-4192-9266-a4505d70fed5 None admin] PATH_INFO: `/v3/users/56357eda81124e6483a36803c6030ddf` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:47.457329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3514b12-72a8-4192-9266-a4505d70fed5 None admin] RBAC: Authorizing `identity:get_user(user_id=56357eda81124e6483a36803c6030ddf)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:47.458762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3514b12-72a8-4192-9266-a4505d70fed5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:47.460387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 582/1158] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:54:47 2022] GET /identity/v3/users/56357eda81124e6483a36803c6030ddf => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:47.466808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfc082-bf2f-4b17-9098-f059c845efd7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:47.506878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:47.566329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfc082-bf2f-4b17-9098-f059c845efd7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:47.599109 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a816bd1-5d41-4187-80c2-ffa380f68008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:47.599725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a816bd1-5d41-4187-80c2-ffa380f68008 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:47.600068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a816bd1-5d41-4187-80c2-ffa380f68008 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:47.600344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a816bd1-5d41-4187-80c2-ffa380f68008 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:47.665353 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a816bd1-5d41-4187-80c2-ffa380f68008 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:47.666728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a816bd1-5d41-4187-80c2-ffa380f68008 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:47.703985 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:47.779103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 577/1159] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:54:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:54:47.787423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3514b12-72a8-4192-9266-a4505d70fed5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:47.884374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3514b12-72a8-4192-9266-a4505d70fed5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:47.900144 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:47.916857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d04c1-6379-4eb0-9af3-7e0d410658b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:47.917454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d04c1-6379-4eb0-9af3-7e0d410658b6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:47.917738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d04c1-6379-4eb0-9af3-7e0d410658b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:47.918013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d04c1-6379-4eb0-9af3-7e0d410658b6 None admin] PATH_INFO: `/v3/projects/93084177668949398520cd51f07d74dc` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:47.919241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d04c1-6379-4eb0-9af3-7e0d410658b6 None admin] RBAC: Authorizing `identity:get_project(project_id=93084177668949398520cd51f07d74dc)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:47.920668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d04c1-6379-4eb0-9af3-7e0d410658b6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:47.922356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 583/1160] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:54:47 2022] GET /identity/v3/projects/93084177668949398520cd51f07d74dc => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:47.930494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a816bd1-5d41-4187-80c2-ffa380f68008 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:48.028106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a816bd1-5d41-4187-80c2-ffa380f68008 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:48.061885 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9564701-2e05-409c-b70c-904fb0b2841a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:48.062494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9564701-2e05-409c-b70c-904fb0b2841a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:48.062778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9564701-2e05-409c-b70c-904fb0b2841a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:48.063058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9564701-2e05-409c-b70c-904fb0b2841a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:48.063843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9564701-2e05-409c-b70c-904fb0b2841a None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=93084177668949398520cd51f07d74dc, user.id=56357eda81124e6483a36803c6030ddf) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:48.064152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9564701-2e05-409c-b70c-904fb0b2841a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:48.065622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9564701-2e05-409c-b70c-904fb0b2841a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:48.071455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 578/1161] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:54:47 2022] GET /identity/v3/role_assignments?user.id=56357eda81124e6483a36803c6030ddf&scope.project.id=93084177668949398520cd51f07d74dc&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:50.096326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea563a-5b9a-4cd0-8733-e8a0b1892bf7 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:50.096861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea563a-5b9a-4cd0-8733-e8a0b1892bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:50.096861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea563a-5b9a-4cd0-8733-e8a0b1892bf7 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:50.098299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 584/1162] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:50.103601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf8c5f-3f3f-4944-a596-87c706912e18 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:50.103995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf8c5f-3f3f-4944-a596-87c706912e18 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:50.104152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf8c5f-3f3f-4944-a596-87c706912e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:50.111105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:50.123242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-ecbf8c5f-3f3f-4944-a596-87c706912e18 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:50.124438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ecbf8c5f-3f3f-4944-a596-87c706912e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:50.189050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 579/1163] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:50.210785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea563a-5b9a-4cd0-8733-e8a0b1892bf7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:50.313938 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea563a-5b9a-4cd0-8733-e8a0b1892bf7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:50.346782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc5e60-54c9-4a14-82e1-37af4a17cc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:50.347329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc5e60-54c9-4a14-82e1-37af4a17cc83 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:50.347614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc5e60-54c9-4a14-82e1-37af4a17cc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:50.347898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc5e60-54c9-4a14-82e1-37af4a17cc83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:50.397205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:50.413054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc5e60-54c9-4a14-82e1-37af4a17cc83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:50.414788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc5e60-54c9-4a14-82e1-37af4a17cc83 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:50.512357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 585/1164] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:54:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:54:50.518342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf8c5f-3f3f-4944-a596-87c706912e18 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:50.615350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf8c5f-3f3f-4944-a596-87c706912e18 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:50.644557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:50.649460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33227153-667f-4878-a917-8f8411ede01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:50.650103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33227153-667f-4878-a917-8f8411ede01d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:50.650387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33227153-667f-4878-a917-8f8411ede01d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:50.650665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33227153-667f-4878-a917-8f8411ede01d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:50.657801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33227153-667f-4878-a917-8f8411ede01d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:50.659874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33227153-667f-4878-a917-8f8411ede01d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-33227153-667f-4878-a917-8f8411ede01d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:54:50.671170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:54:50.672438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:54:50.673366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 580/1165] 10.222.0.86 () {56 vars in 1117 bytes} [Thu Jul 28 17:54:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:54:50.678392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc5e60-54c9-4a14-82e1-37af4a17cc83 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:50.771876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:50.787699 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc5e60-54c9-4a14-82e1-37af4a17cc83 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:50.821060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5a524-c649-4e61-a14c-c30d8d14b32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:50.821671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5a524-c649-4e61-a14c-c30d8d14b32c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:50.821953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5a524-c649-4e61-a14c-c30d8d14b32c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:50.822227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5a524-c649-4e61-a14c-c30d8d14b32c None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:50.822908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5a524-c649-4e61-a14c-c30d8d14b32c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:50.823213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5a524-c649-4e61-a14c-c30d8d14b32c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:50.824569 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5a524-c649-4e61-a14c-c30d8d14b32c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:50.831462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 586/1166] 10.222.0.86 () {56 vars in 1115 bytes} [Thu Jul 28 17:54:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:52.789422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a68e4-828d-42fc-b6ce-ba4e9a824420 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:52.789942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a68e4-828d-42fc-b6ce-ba4e9a824420 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:52.789942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a68e4-828d-42fc-b6ce-ba4e9a824420 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:52.791277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 581/1167] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:52.796773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27536b0c-f7d3-42ea-a99d-288b6cad3019 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:52.797209 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27536b0c-f7d3-42ea-a99d-288b6cad3019 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:52.797361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27536b0c-f7d3-42ea-a99d-288b6cad3019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:52.815715 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-27536b0c-f7d3-42ea-a99d-288b6cad3019 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:52.816889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-27536b0c-f7d3-42ea-a99d-288b6cad3019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:52.880423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 587/1168] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:52.902898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a68e4-828d-42fc-b6ce-ba4e9a824420 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:53.001770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a68e4-828d-42fc-b6ce-ba4e9a824420 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:53.034052 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fce527-3d68-4ab8-bec0-b4e4d8e5006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:53.034651 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fce527-3d68-4ab8-bec0-b4e4d8e5006d None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:53.034934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fce527-3d68-4ab8-bec0-b4e4d8e5006d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:53.035210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fce527-3d68-4ab8-bec0-b4e4d8e5006d None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:53.035924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fce527-3d68-4ab8-bec0-b4e4d8e5006d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:53.037373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fce527-3d68-4ab8-bec0-b4e4d8e5006d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:53.125227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 582/1169] 10.222.0.86 () {60 vars in 1138 bytes} [Thu Jul 28 17:54:52 2022] POST /identity/v3/domains => generated 267 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:54:55.128624 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f739081-a1e1-414c-8d88-f149efbfe6f2 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:55.129218 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f739081-a1e1-414c-8d88-f149efbfe6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:55.129218 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f739081-a1e1-414c-8d88-f149efbfe6f2 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:55.130572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 588/1170] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:55.136136 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc2459-53e8-4117-b63d-4a7a642df108 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:55.136542 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc2459-53e8-4117-b63d-4a7a642df108 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:55.136698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc2459-53e8-4117-b63d-4a7a642df108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:55.155057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6ebc2459-53e8-4117-b63d-4a7a642df108 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:55.156247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6ebc2459-53e8-4117-b63d-4a7a642df108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:55.202723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:55.221622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 583/1171] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:55.244405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f739081-a1e1-414c-8d88-f149efbfe6f2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:55.322074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:55.343881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f739081-a1e1-414c-8d88-f149efbfe6f2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:55.377231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0afddf-a668-41d0-a8a1-4f3c14dcd416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:55.377861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0afddf-a668-41d0-a8a1-4f3c14dcd416 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:55.378147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0afddf-a668-41d0-a8a1-4f3c14dcd416 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:55.378426 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0afddf-a668-41d0-a8a1-4f3c14dcd416 None admin] PATH_INFO: `/v3/domains/2c2c36a9d34b426584bde7d08bcec3cb` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:55.389194 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0afddf-a668-41d0-a8a1-4f3c14dcd416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2c2c36a9d34b426584bde7d08bcec3cb)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:55.390611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0afddf-a668-41d0-a8a1-4f3c14dcd416 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:55.392218 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 589/1172] 10.222.0.86 () {56 vars in 1183 bytes} [Thu Jul 28 17:54:55 2022] GET /identity/v3/domains/2c2c36a9d34b426584bde7d08bcec3cb => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:55.398056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc2459-53e8-4117-b63d-4a7a642df108 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:55.495265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc2459-53e8-4117-b63d-4a7a642df108 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:55.528686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f45b8-7e0b-423c-b3b1-abb21183a6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:55.529329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f45b8-7e0b-423c-b3b1-abb21183a6e1 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:55.529617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f45b8-7e0b-423c-b3b1-abb21183a6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:55.529895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f45b8-7e0b-423c-b3b1-abb21183a6e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:55.530590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f45b8-7e0b-423c-b3b1-abb21183a6e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:55.532018 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f45b8-7e0b-423c-b3b1-abb21183a6e1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:55.622117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 584/1173] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:54:55 2022] POST /identity/v3/projects => generated 361 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:57.645003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db7db3-2df2-4f47-ba86-1512f9badd3d None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:57.645520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db7db3-2df2-4f47-ba86-1512f9badd3d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:57.645634 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db7db3-2df2-4f47-ba86-1512f9badd3d None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:57.646994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 590/1174] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:54:57.652789 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742440-3e26-4c5a-89d0-3f177fbb0b84 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:57.653259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742440-3e26-4c5a-89d0-3f177fbb0b84 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:57.653362 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742440-3e26-4c5a-89d0-3f177fbb0b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:57.671530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-fe742440-3e26-4c5a-89d0-3f177fbb0b84 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:57.672756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fe742440-3e26-4c5a-89d0-3f177fbb0b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:57.698360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:57.738206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 585/1175] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:54:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:54:57.763245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db7db3-2df2-4f47-ba86-1512f9badd3d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:57.863805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db7db3-2df2-4f47-ba86-1512f9badd3d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:57.896494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c27a7a-3c6d-40e5-a578-209b1bba5a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:57.897155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27a7a-3c6d-40e5-a578-209b1bba5a40 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:57.897444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27a7a-3c6d-40e5-a578-209b1bba5a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:57.897725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27a7a-3c6d-40e5-a578-209b1bba5a40 None admin] PATH_INFO: `/v3/domains/2c2c36a9d34b426584bde7d08bcec3cb` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:57.898888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c27a7a-3c6d-40e5-a578-209b1bba5a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2c2c36a9d34b426584bde7d08bcec3cb)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:57.900321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c27a7a-3c6d-40e5-a578-209b1bba5a40 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:57.902558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 591/1176] 10.222.0.86 () {56 vars in 1183 bytes} [Thu Jul 28 17:54:57 2022] GET /identity/v3/domains/2c2c36a9d34b426584bde7d08bcec3cb => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:57.908218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe742440-3e26-4c5a-89d0-3f177fbb0b84 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:58.004717 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe742440-3e26-4c5a-89d0-3f177fbb0b84 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:58.037836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2679835-68be-459d-97ad-d5c932394fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:58.038447 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2679835-68be-459d-97ad-d5c932394fa2 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:58.038731 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2679835-68be-459d-97ad-d5c932394fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:58.039010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2679835-68be-459d-97ad-d5c932394fa2 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:58.039699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2679835-68be-459d-97ad-d5c932394fa2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:58.041155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2679835-68be-459d-97ad-d5c932394fa2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:58.129539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 586/1177] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:54:57 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:55:00.176013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f25ff3-c3d2-4051-b8d9-58cde18432c0 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:00.176540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f25ff3-c3d2-4051-b8d9-58cde18432c0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:00.176540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f25ff3-c3d2-4051-b8d9-58cde18432c0 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:00.178138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 592/1178] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:00.183363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890cd4c-aec5-40c6-b7a0-a4b07f31f001 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:00.183786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890cd4c-aec5-40c6-b7a0-a4b07f31f001 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:00.183933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890cd4c-aec5-40c6-b7a0-a4b07f31f001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:00.202352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7890cd4c-aec5-40c6-b7a0-a4b07f31f001 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:00.203578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7890cd4c-aec5-40c6-b7a0-a4b07f31f001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:00.217992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:00.269331 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 587/1179] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:55:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:55:00.293079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f25ff3-c3d2-4051-b8d9-58cde18432c0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:00.302842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:00.392825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f25ff3-c3d2-4051-b8d9-58cde18432c0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:00.427282 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a02840-1228-4993-8957-f28ef6901418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:00.427931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a02840-1228-4993-8957-f28ef6901418 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:00.428208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a02840-1228-4993-8957-f28ef6901418 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:00.428478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a02840-1228-4993-8957-f28ef6901418 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:00.439236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a02840-1228-4993-8957-f28ef6901418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f9a02840-1228-4993-8957-f28ef6901418 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:55:00.444221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:55:00.445370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:55:00.446491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 593/1180] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:55:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:55:00.451832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890cd4c-aec5-40c6-b7a0-a4b07f31f001 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:00.553821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890cd4c-aec5-40c6-b7a0-a4b07f31f001 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:00.587925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1328d-5224-4ca7-9612-5970e75b0749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:00.588548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1328d-5224-4ca7-9612-5970e75b0749 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:00.588830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1328d-5224-4ca7-9612-5970e75b0749 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:00.589140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1328d-5224-4ca7-9612-5970e75b0749 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:00.589864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1328d-5224-4ca7-9612-5970e75b0749 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:55:00.590170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1328d-5224-4ca7-9612-5970e75b0749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:00.591539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1328d-5224-4ca7-9612-5970e75b0749 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:00.599416 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 588/1181] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:55:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:55:00.605228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a02840-1228-4993-8957-f28ef6901418 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:00.668707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:00.702926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a02840-1228-4993-8957-f28ef6901418 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:00.734412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0a424-c8ac-400f-97c8-e3dbae5a1c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:00.735046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0a424-c8ac-400f-97c8-e3dbae5a1c5f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:00.735330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0a424-c8ac-400f-97c8-e3dbae5a1c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:00.735607 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0a424-c8ac-400f-97c8-e3dbae5a1c5f None admin] PATH_INFO: `/v3/users/e307c339bd4544b4b426564627a1b7c7` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:00.749160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0a424-c8ac-400f-97c8-e3dbae5a1c5f None admin] RBAC: Authorizing `identity:get_user(user_id=e307c339bd4544b4b426564627a1b7c7)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:00.750596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0a424-c8ac-400f-97c8-e3dbae5a1c5f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:00.752233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 594/1182] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:55:00 2022] GET /identity/v3/users/e307c339bd4544b4b426564627a1b7c7 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:55:00.758035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1328d-5224-4ca7-9612-5970e75b0749 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:00.773616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:00.857475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1328d-5224-4ca7-9612-5970e75b0749 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:00.890061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07942f24-9f70-41f4-abfa-88277f0adb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:00.890669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07942f24-9f70-41f4-abfa-88277f0adb3e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:00.890952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07942f24-9f70-41f4-abfa-88277f0adb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:00.891228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07942f24-9f70-41f4-abfa-88277f0adb3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:00.956088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07942f24-9f70-41f4-abfa-88277f0adb3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:00.957492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07942f24-9f70-41f4-abfa-88277f0adb3e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:00.970271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:01.052565 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 589/1183] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:55:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:55:01.059001 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0a424-c8ac-400f-97c8-e3dbae5a1c5f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:01.155686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0a424-c8ac-400f-97c8-e3dbae5a1c5f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:01.178264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:01.188871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c34070-5c6a-4a17-9aa3-e6386cb10d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:01.189539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c34070-5c6a-4a17-9aa3-e6386cb10d97 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:01.189824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c34070-5c6a-4a17-9aa3-e6386cb10d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:01.190102 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c34070-5c6a-4a17-9aa3-e6386cb10d97 None admin] PATH_INFO: `/v3/projects/85af565c4752463c8d448a1455350de7` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:01.191283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c34070-5c6a-4a17-9aa3-e6386cb10d97 None admin] RBAC: Authorizing `identity:get_project(project_id=85af565c4752463c8d448a1455350de7)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:01.192744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c34070-5c6a-4a17-9aa3-e6386cb10d97 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:01.194460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 595/1184] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:55:01 2022] GET /identity/v3/projects/85af565c4752463c8d448a1455350de7 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:55:01.201208 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07942f24-9f70-41f4-abfa-88277f0adb3e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:01.296160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07942f24-9f70-41f4-abfa-88277f0adb3e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:01.309161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:01.329378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14298c5f-ce7a-4971-bc59-39a96653cee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:01.329983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14298c5f-ce7a-4971-bc59-39a96653cee3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:01.330266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14298c5f-ce7a-4971-bc59-39a96653cee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:01.330540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14298c5f-ce7a-4971-bc59-39a96653cee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:01.331323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14298c5f-ce7a-4971-bc59-39a96653cee3 None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=85af565c4752463c8d448a1455350de7, user.id=e307c339bd4544b4b426564627a1b7c7) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:55:01.331635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14298c5f-ce7a-4971-bc59-39a96653cee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:01.333097 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14298c5f-ce7a-4971-bc59-39a96653cee3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:01.338343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 590/1185] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:55:01 2022] GET /identity/v3/role_assignments?user.id=e307c339bd4544b4b426564627a1b7c7&scope.project.id=85af565c4752463c8d448a1455350de7&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:55:03.307554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b160351-2866-455d-8b32-d0cc7cbe4b7a None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:03.308124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b160351-2866-455d-8b32-d0cc7cbe4b7a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:03.308124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b160351-2866-455d-8b32-d0cc7cbe4b7a None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:03.309627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 596/1186] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:55:03.314897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c35ac-41fe-4fea-b60a-c86b2094ff40 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:03.315299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c35ac-41fe-4fea-b60a-c86b2094ff40 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:03.315458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c35ac-41fe-4fea-b60a-c86b2094ff40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:03.333835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-941c35ac-41fe-4fea-b60a-c86b2094ff40 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:03.335031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-941c35ac-41fe-4fea-b60a-c86b2094ff40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:03.400317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 591/1187] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:55:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:55:03.423350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b160351-2866-455d-8b32-d0cc7cbe4b7a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:03.521374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b160351-2866-455d-8b32-d0cc7cbe4b7a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:03.558114 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0a120-9125-43fb-b427-e33094b84e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:03.558682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0a120-9125-43fb-b427-e33094b84e7f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:03.558960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0a120-9125-43fb-b427-e33094b84e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:03.559233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0a120-9125-43fb-b427-e33094b84e7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:03.569707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0a120-9125-43fb-b427-e33094b84e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a6e0a120-9125-43fb-b427-e33094b84e7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:55:03.575998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:55:03.577167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:55:03.578251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 597/1188] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:55:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:55:03.583267 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c35ac-41fe-4fea-b60a-c86b2094ff40 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:03.678594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c35ac-41fe-4fea-b60a-c86b2094ff40 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:03.686091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:03.711861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf199efa-c44a-4b22-a513-4e67570958d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:03.712464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf199efa-c44a-4b22-a513-4e67570958d1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:03.718263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf199efa-c44a-4b22-a513-4e67570958d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:03.718263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf199efa-c44a-4b22-a513-4e67570958d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:03.718263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf199efa-c44a-4b22-a513-4e67570958d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:55:03.718263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf199efa-c44a-4b22-a513-4e67570958d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:03.718263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf199efa-c44a-4b22-a513-4e67570958d1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:03.723078 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 592/1189] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:55:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:55:03.728781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0a120-9125-43fb-b427-e33094b84e7f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:03.781899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:03.828362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0a120-9125-43fb-b427-e33094b84e7f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:03.861516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ff141-dd69-4914-aee0-c587d4177417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:03.862138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ff141-dd69-4914-aee0-c587d4177417 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:03.862415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ff141-dd69-4914-aee0-c587d4177417 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:03.862688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ff141-dd69-4914-aee0-c587d4177417 None admin] PATH_INFO: `/v3/users/e307c339bd4544b4b426564627a1b7c7` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:03.863832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ff141-dd69-4914-aee0-c587d4177417 None admin] RBAC: Authorizing `identity:get_user(user_id=e307c339bd4544b4b426564627a1b7c7)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:03.865356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ff141-dd69-4914-aee0-c587d4177417 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:03.866922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 598/1190] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:55:03 2022] GET /identity/v3/users/e307c339bd4544b4b426564627a1b7c7 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:55:03.873693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf199efa-c44a-4b22-a513-4e67570958d1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:03.969829 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf199efa-c44a-4b22-a513-4e67570958d1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:04.003307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ca951-b217-4ce7-8dd6-2d4d039219a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:04.003942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ca951-b217-4ce7-8dd6-2d4d039219a1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:04.004229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ca951-b217-4ce7-8dd6-2d4d039219a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:04.004505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ca951-b217-4ce7-8dd6-2d4d039219a1 None admin] PATH_INFO: `/v3/projects/85af565c4752463c8d448a1455350de7` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:04.005686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ca951-b217-4ce7-8dd6-2d4d039219a1 None admin] RBAC: Authorizing `identity:get_project(project_id=85af565c4752463c8d448a1455350de7)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:04.007105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ca951-b217-4ce7-8dd6-2d4d039219a1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:04.008748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 593/1191] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:55:03 2022] GET /identity/v3/projects/85af565c4752463c8d448a1455350de7 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:55:04.014679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ff141-dd69-4914-aee0-c587d4177417 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:04.110995 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ff141-dd69-4914-aee0-c587d4177417 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:04.131369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:04.145141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae6a0c-7320-4e98-872e-5d3ac4bfc595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:04.145787 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6a0c-7320-4e98-872e-5d3ac4bfc595 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:04.146068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6a0c-7320-4e98-872e-5d3ac4bfc595 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:04.146346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6a0c-7320-4e98-872e-5d3ac4bfc595 None admin] PATH_INFO: `/v3/projects/85af565c4752463c8d448a1455350de7/users/e307c339bd4544b4b426564627a1b7c7/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:04.148283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae6a0c-7320-4e98-872e-5d3ac4bfc595 None admin] RBAC: Authorizing `identity:create_grant(project_id=85af565c4752463c8d448a1455350de7, user_id=e307c339bd4544b4b426564627a1b7c7, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:04.149744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae6a0c-7320-4e98-872e-5d3ac4bfc595 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:04.232459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 599/1192] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:55:04 2022] PUT /identity/v3/projects/85af565c4752463c8d448a1455350de7/users/e307c339bd4544b4b426564627a1b7c7/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:55:06.190212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e47eed-33dc-479e-b4d0-c021d9e2483e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:06.190780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e47eed-33dc-479e-b4d0-c021d9e2483e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:06.190780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e47eed-33dc-479e-b4d0-c021d9e2483e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:06.192161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 594/1193] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:55:06.197373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fa788-1b27-49d6-b416-28f18bfb62f7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:06.197820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fa788-1b27-49d6-b416-28f18bfb62f7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:06.198221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fa788-1b27-49d6-b416-28f18bfb62f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:06.216194 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-227fa788-1b27-49d6-b416-28f18bfb62f7 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:55:06.217420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-227fa788-1b27-49d6-b416-28f18bfb62f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:06.281792 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 600/1194] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:55:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:55:06.304946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e47eed-33dc-479e-b4d0-c021d9e2483e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:06.340706 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:06.406194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e47eed-33dc-479e-b4d0-c021d9e2483e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:06.439411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2ed44-109f-4df1-ad9d-aafd8b29739c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:06.440044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2ed44-109f-4df1-ad9d-aafd8b29739c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:06.440374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2ed44-109f-4df1-ad9d-aafd8b29739c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:06.440603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2ed44-109f-4df1-ad9d-aafd8b29739c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:06.450835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2ed44-109f-4df1-ad9d-aafd8b29739c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-83d2ed44-109f-4df1-ad9d-aafd8b29739c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:55:06.455691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:55:06.456794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:55:06.457894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 595/1195] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:55:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:55:06.462844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fa788-1b27-49d6-b416-28f18bfb62f7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:06.562757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fa788-1b27-49d6-b416-28f18bfb62f7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:06.572987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:06.595572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1784a-fced-41d1-b59b-f262fcb93627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:06.596186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1784a-fced-41d1-b59b-f262fcb93627 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:06.596468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1784a-fced-41d1-b59b-f262fcb93627 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:06.596744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1784a-fced-41d1-b59b-f262fcb93627 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:06.597496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1784a-fced-41d1-b59b-f262fcb93627 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:55:06.597804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1784a-fced-41d1-b59b-f262fcb93627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:06.599156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1784a-fced-41d1-b59b-f262fcb93627 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:06.608103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 601/1196] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:55:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:55:06.613952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2ed44-109f-4df1-ad9d-aafd8b29739c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:06.705507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:06.711154 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2ed44-109f-4df1-ad9d-aafd8b29739c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:06.744260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038eb4b-6135-4b8f-9a71-debd5daa682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:06.744925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038eb4b-6135-4b8f-9a71-debd5daa682a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:06.745584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038eb4b-6135-4b8f-9a71-debd5daa682a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:06.745859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038eb4b-6135-4b8f-9a71-debd5daa682a None admin] PATH_INFO: `/v3/users/e307c339bd4544b4b426564627a1b7c7` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:06.747004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038eb4b-6135-4b8f-9a71-debd5daa682a None admin] RBAC: Authorizing `identity:get_user(user_id=e307c339bd4544b4b426564627a1b7c7)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:06.748683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038eb4b-6135-4b8f-9a71-debd5daa682a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:06.750305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 596/1197] 10.222.0.86 () {56 vars in 1177 bytes} [Thu Jul 28 17:55:06 2022] GET /identity/v3/users/e307c339bd4544b4b426564627a1b7c7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:55:06.756245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1784a-fced-41d1-b59b-f262fcb93627 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:06.866516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1784a-fced-41d1-b59b-f262fcb93627 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:06.899707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867bad1-5f3c-4dac-9ddb-37a4af5337da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:06.900322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867bad1-5f3c-4dac-9ddb-37a4af5337da None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:06.901206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867bad1-5f3c-4dac-9ddb-37a4af5337da None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:06.901487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867bad1-5f3c-4dac-9ddb-37a4af5337da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:06.940022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:06.966170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867bad1-5f3c-4dac-9ddb-37a4af5337da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:06.968016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867bad1-5f3c-4dac-9ddb-37a4af5337da None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:07.062069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 602/1198] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:55:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:55:07.068000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038eb4b-6135-4b8f-9a71-debd5daa682a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:07.164605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038eb4b-6135-4b8f-9a71-debd5daa682a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:07.196858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707afc6-c5ec-46b8-9fd3-d8827609afc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:07.198051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707afc6-c5ec-46b8-9fd3-d8827609afc0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:07.198316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707afc6-c5ec-46b8-9fd3-d8827609afc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:07.198586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707afc6-c5ec-46b8-9fd3-d8827609afc0 None admin] PATH_INFO: `/v3/projects/85af565c4752463c8d448a1455350de7` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:07.199738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707afc6-c5ec-46b8-9fd3-d8827609afc0 None admin] RBAC: Authorizing `identity:get_project(project_id=85af565c4752463c8d448a1455350de7)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:07.201188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707afc6-c5ec-46b8-9fd3-d8827609afc0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:07.202827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 597/1199] 10.222.0.86 () {56 vars in 1186 bytes} [Thu Jul 28 17:55:07 2022] GET /identity/v3/projects/85af565c4752463c8d448a1455350de7 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:55:07.208873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867bad1-5f3c-4dac-9ddb-37a4af5337da None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:07.277187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:07.304786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867bad1-5f3c-4dac-9ddb-37a4af5337da None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:07.339170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ef006-8ee8-40e4-ada0-c8101e4a5597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:07.339774 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ef006-8ee8-40e4-ada0-c8101e4a5597 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:07.340056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ef006-8ee8-40e4-ada0-c8101e4a5597 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:07.340334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ef006-8ee8-40e4-ada0-c8101e4a5597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:07.341146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ef006-8ee8-40e4-ada0-c8101e4a5597 None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=85af565c4752463c8d448a1455350de7, user.id=e307c339bd4544b4b426564627a1b7c7) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:55:07.341460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ef006-8ee8-40e4-ada0-c8101e4a5597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:07.342880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ef006-8ee8-40e4-ada0-c8101e4a5597 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:07.348558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 603/1200] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:55:07 2022] GET /identity/v3/role_assignments?user.id=e307c339bd4544b4b426564627a1b7c7&scope.project.id=85af565c4752463c8d448a1455350de7&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:55:17.192851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=84 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:17.345946 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=32 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:19.504547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbdca9-b536-4c3f-8946-ec4706246316 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.505148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbdca9-b536-4c3f-8946-ec4706246316 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.505148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbdca9-b536-4c3f-8946-ec4706246316 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.506486 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 598/1201] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:19.511506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2161f3f-bd34-4aad-8dbf-9f6184499d3a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.511934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2161f3f-bd34-4aad-8dbf-9f6184499d3a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.512063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2161f3f-bd34-4aad-8dbf-9f6184499d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.529886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b2161f3f-bd34-4aad-8dbf-9f6184499d3a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:19.531067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b2161f3f-bd34-4aad-8dbf-9f6184499d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:19.599852 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 604/1202] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:56:19.625791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbdca9-b536-4c3f-8946-ec4706246316 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:19.725305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbdca9-b536-4c3f-8946-ec4706246316 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:19.759579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b0a00-ec58-4fd9-bf8a-dad3b8d2b070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:19.760597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0a00-ec58-4fd9-bf8a-dad3b8d2b070 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.760884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0a00-ec58-4fd9-bf8a-dad3b8d2b070 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.761193 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0a00-ec58-4fd9-bf8a-dad3b8d2b070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.772376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b0a00-ec58-4fd9-bf8a-dad3b8d2b070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:19.773875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b0a00-ec58-4fd9-bf8a-dad3b8d2b070 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-414b0a00-ec58-4fd9-bf8a-dad3b8d2b070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:19.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:19.790092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:19.791378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:19.791378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:19.791378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:19.791378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:19.791378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 599/1203] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:19.796271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2161f3f-bd34-4aad-8dbf-9f6184499d3a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:19.892365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2161f3f-bd34-4aad-8dbf-9f6184499d3a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:19.920832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:19.926514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c489a2-760b-4035-b2a3-b542e0da8f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:19.927235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c489a2-760b-4035-b2a3-b542e0da8f25 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.927516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c489a2-760b-4035-b2a3-b542e0da8f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.927789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c489a2-760b-4035-b2a3-b542e0da8f25 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.928474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c489a2-760b-4035-b2a3-b542e0da8f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:19.928776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c489a2-760b-4035-b2a3-b542e0da8f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:19.930251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c489a2-760b-4035-b2a3-b542e0da8f25 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:19.941275 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 605/1204] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:19.947042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b0a00-ec58-4fd9-bf8a-dad3b8d2b070 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:20.001673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:20.051096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b0a00-ec58-4fd9-bf8a-dad3b8d2b070 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:20.084462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecd136-da71-47bc-93b4-554ed13104d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:20.085116 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd136-da71-47bc-93b4-554ed13104d0 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:20.085404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd136-da71-47bc-93b4-554ed13104d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:20.085679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd136-da71-47bc-93b4-554ed13104d0 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:20.086840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecd136-da71-47bc-93b4-554ed13104d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:20.088299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecd136-da71-47bc-93b4-554ed13104d0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:20.199963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 600/1205] 10.222.0.86 () {60 vars in 1132 bytes} [Thu Jul 28 17:56:19 2022] POST /identity/v3/users => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:22.245771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbe818-9dfb-49a9-b57f-8627eeb98129 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.246302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbe818-9dfb-49a9-b57f-8627eeb98129 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.246302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbe818-9dfb-49a9-b57f-8627eeb98129 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.247684 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 606/1206] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:22.252948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c902378-b8c8-4bba-ac7d-00bdf3656311 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.253377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c902378-b8c8-4bba-ac7d-00bdf3656311 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.253537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c902378-b8c8-4bba-ac7d-00bdf3656311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.272107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-4c902378-b8c8-4bba-ac7d-00bdf3656311 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:22.273341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4c902378-b8c8-4bba-ac7d-00bdf3656311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:22.340066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 601/1207] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:56:22.363189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbe818-9dfb-49a9-b57f-8627eeb98129 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.465022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbe818-9dfb-49a9-b57f-8627eeb98129 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.499017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c5054-0c41-42a4-bfad-46bcd7b9c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.499647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c5054-0c41-42a4-bfad-46bcd7b9c2ff None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.499923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c5054-0c41-42a4-bfad-46bcd7b9c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.500196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c5054-0c41-42a4-bfad-46bcd7b9c2ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.510581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c5054-0c41-42a4-bfad-46bcd7b9c2ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-519c5054-0c41-42a4-bfad-46bcd7b9c2ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:56:22.515632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:22.516769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:22.518994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 607/1208] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:56:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:22.523408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c902378-b8c8-4bba-ac7d-00bdf3656311 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.563367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:22.621643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c902378-b8c8-4bba-ac7d-00bdf3656311 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.654645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada92501-ed10-4a7c-b5ff-fc6d2c97878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.655264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada92501-ed10-4a7c-b5ff-fc6d2c97878a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.655549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada92501-ed10-4a7c-b5ff-fc6d2c97878a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.655825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada92501-ed10-4a7c-b5ff-fc6d2c97878a None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.656550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada92501-ed10-4a7c-b5ff-fc6d2c97878a None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:22.656858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada92501-ed10-4a7c-b5ff-fc6d2c97878a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.658263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada92501-ed10-4a7c-b5ff-fc6d2c97878a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.666744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 602/1209] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:56:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:22.672856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c5054-0c41-42a4-bfad-46bcd7b9c2ff None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.722069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:22.770787 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c5054-0c41-42a4-bfad-46bcd7b9c2ff None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.802585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bdb48-4609-47b8-a8df-ec9fece3e749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.803220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bdb48-4609-47b8-a8df-ec9fece3e749 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.803500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bdb48-4609-47b8-a8df-ec9fece3e749 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.803774 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bdb48-4609-47b8-a8df-ec9fece3e749 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.814501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bdb48-4609-47b8-a8df-ec9fece3e749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.815936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bdb48-4609-47b8-a8df-ec9fece3e749 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-b77bdb48-4609-47b8-a8df-ec9fece3e749 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:22.831025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:22.832118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:22.833351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:22.833351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:22.833351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:22.833351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:22.833351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 608/1210] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:22.838226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada92501-ed10-4a7c-b5ff-fc6d2c97878a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.934617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:22.937455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada92501-ed10-4a7c-b5ff-fc6d2c97878a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.970570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe92b4d-72ba-46a1-b622-7062ca641a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.971275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe92b4d-72ba-46a1-b622-7062ca641a51 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.971558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe92b4d-72ba-46a1-b622-7062ca641a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.971831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe92b4d-72ba-46a1-b622-7062ca641a51 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.972515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe92b4d-72ba-46a1-b622-7062ca641a51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:22.972816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe92b4d-72ba-46a1-b622-7062ca641a51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.974202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe92b4d-72ba-46a1-b622-7062ca641a51 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.986262 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 603/1211] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:22.991953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bdb48-4609-47b8-a8df-ec9fece3e749 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:23.092510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bdb48-4609-47b8-a8df-ec9fece3e749 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:23.094210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:23.127446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5a9ce-5d87-449f-a0f7-f95c2d9ce229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:23.128086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5a9ce-5d87-449f-a0f7-f95c2d9ce229 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:23.128367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5a9ce-5d87-449f-a0f7-f95c2d9ce229 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:23.128642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5a9ce-5d87-449f-a0f7-f95c2d9ce229 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:23.135870 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5a9ce-5d87-449f-a0f7-f95c2d9ce229 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:23.137336 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5a9ce-5d87-449f-a0f7-f95c2d9ce229 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-9fd5a9ce-5d87-449f-a0f7-f95c2d9ce229 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:56:23.148136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:23.149288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:23.151083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:23.151083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:23.151083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:56:23.151083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:56:23.151083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:56:23.151083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:56:23.151083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:56:23.151083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:56:23.151083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:23.151083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:23.151083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 609/1212] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:23.155681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe92b4d-72ba-46a1-b622-7062ca641a51 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:23.253710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe92b4d-72ba-46a1-b622-7062ca641a51 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:23.287426 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65109906-93bc-4448-a5fe-18d567990b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:23.288041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109906-93bc-4448-a5fe-18d567990b04 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:23.288328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109906-93bc-4448-a5fe-18d567990b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:23.288607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109906-93bc-4448-a5fe-18d567990b04 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:23.289450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65109906-93bc-4448-a5fe-18d567990b04 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:23.289752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65109906-93bc-4448-a5fe-18d567990b04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:23.291216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65109906-93bc-4448-a5fe-18d567990b04 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:23.308159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 604/1213] 10.222.0.86 () {56 vars in 1143 bytes} [Thu Jul 28 17:56:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:23.314250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5a9ce-5d87-449f-a0f7-f95c2d9ce229 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:23.414501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5a9ce-5d87-449f-a0f7-f95c2d9ce229 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:23.447716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e126a-a9e5-4cc6-88e5-79feb7b5873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:23.448336 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e126a-a9e5-4cc6-88e5-79feb7b5873b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:23.448616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e126a-a9e5-4cc6-88e5-79feb7b5873b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:23.448889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e126a-a9e5-4cc6-88e5-79feb7b5873b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:23.477781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:23.514639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e126a-a9e5-4cc6-88e5-79feb7b5873b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:23.516071 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e126a-a9e5-4cc6-88e5-79feb7b5873b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:23.613712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 610/1214] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:56:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:56:23.620411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65109906-93bc-4448-a5fe-18d567990b04 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:23.639861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:23.719332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65109906-93bc-4448-a5fe-18d567990b04 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:23.752916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1825f2-e30f-4c20-a642-6ba8b25a92da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:23.753592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1825f2-e30f-4c20-a642-6ba8b25a92da None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:23.753874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1825f2-e30f-4c20-a642-6ba8b25a92da None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:23.754293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1825f2-e30f-4c20-a642-6ba8b25a92da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:23.768038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1825f2-e30f-4c20-a642-6ba8b25a92da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:23.769597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1825f2-e30f-4c20-a642-6ba8b25a92da None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-5f1825f2-e30f-4c20-a642-6ba8b25a92da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:23.783940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:23.785089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:23.786258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:23.786258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:23.786258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:23.786258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:23.786258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 605/1215] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:23.790986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e126a-a9e5-4cc6-88e5-79feb7b5873b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:23.850913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:23.889861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e126a-a9e5-4cc6-88e5-79feb7b5873b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:23.923030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46182497-a594-4e1d-9c48-bf9169f89af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:23.923641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46182497-a594-4e1d-9c48-bf9169f89af0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:23.923926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46182497-a594-4e1d-9c48-bf9169f89af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:23.924203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46182497-a594-4e1d-9c48-bf9169f89af0 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:23.924894 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46182497-a594-4e1d-9c48-bf9169f89af0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:23.925234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46182497-a594-4e1d-9c48-bf9169f89af0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:23.926596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46182497-a594-4e1d-9c48-bf9169f89af0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:23.938206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 611/1216] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:23.944127 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1825f2-e30f-4c20-a642-6ba8b25a92da None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:24.014474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:24.055189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1825f2-e30f-4c20-a642-6ba8b25a92da None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:24.089761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5367daf-b42e-4e81-a0a5-338badd3eb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:24.090460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5367daf-b42e-4e81-a0a5-338badd3eb65 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:24.090822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5367daf-b42e-4e81-a0a5-338badd3eb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:24.091131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5367daf-b42e-4e81-a0a5-338badd3eb65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:24.098077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5367daf-b42e-4e81-a0a5-338badd3eb65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:24.099579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5367daf-b42e-4e81-a0a5-338badd3eb65 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c5367daf-b42e-4e81-a0a5-338badd3eb65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:56:24.111348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:24.112505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:24.117107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:56:24.117107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:56:24.117107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:56:24.117107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:56:24.117107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:56:24.117107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:56:24.117107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:24.117107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:24.117107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 606/1217] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:56:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:24.119525 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46182497-a594-4e1d-9c48-bf9169f89af0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:24.217165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46182497-a594-4e1d-9c48-bf9169f89af0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:24.240361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:24.252363 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067e425-a3f6-4965-a83f-acfd204698cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:24.253006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067e425-a3f6-4965-a83f-acfd204698cf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:24.253294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067e425-a3f6-4965-a83f-acfd204698cf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:24.253573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067e425-a3f6-4965-a83f-acfd204698cf None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:24.254330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067e425-a3f6-4965-a83f-acfd204698cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:24.254650 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067e425-a3f6-4965-a83f-acfd204698cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:24.256094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067e425-a3f6-4965-a83f-acfd204698cf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:24.268564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 612/1218] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:56:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:24.274848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5367daf-b42e-4e81-a0a5-338badd3eb65 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:24.374815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5367daf-b42e-4e81-a0a5-338badd3eb65 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:24.414164 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:24.424595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec0b09-c411-4448-ad22-db4b97bc1bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:24.425261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec0b09-c411-4448-ad22-db4b97bc1bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:24.425546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec0b09-c411-4448-ad22-db4b97bc1bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:24.425823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec0b09-c411-4448-ad22-db4b97bc1bf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:24.426622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec0b09-c411-4448-ad22-db4b97bc1bf4 None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=b20564a83953433290aa3fabf884370c) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:24.426933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec0b09-c411-4448-ad22-db4b97bc1bf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:24.428495 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec0b09-c411-4448-ad22-db4b97bc1bf4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:24.437160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 607/1219] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:56:24 2022] GET /identity/v3/role_assignments?user.id=b20564a83953433290aa3fabf884370c&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:26.425515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a2c7e-22d1-4ca5-910b-c5895e16a8ca None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.425923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a2c7e-22d1-4ca5-910b-c5895e16a8ca None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.426070 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a2c7e-22d1-4ca5-910b-c5895e16a8ca None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.427436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 613/1220] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:26.433229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906532e-041a-4487-86ba-520c200dfcea None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.433682 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906532e-041a-4487-86ba-520c200dfcea None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.433777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906532e-041a-4487-86ba-520c200dfcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.451618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1906532e-041a-4487-86ba-520c200dfcea None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:26.452811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1906532e-041a-4487-86ba-520c200dfcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:26.518719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 608/1221] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:56:26.541330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a2c7e-22d1-4ca5-910b-c5895e16a8ca None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.641599 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a2c7e-22d1-4ca5-910b-c5895e16a8ca None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.676538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfa722-e096-4989-ac90-1d35ae0e6868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.677136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa722-e096-4989-ac90-1d35ae0e6868 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.677423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa722-e096-4989-ac90-1d35ae0e6868 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.677706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa722-e096-4989-ac90-1d35ae0e6868 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.688092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfa722-e096-4989-ac90-1d35ae0e6868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-52dfa722-e096-4989-ac90-1d35ae0e6868 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:56:26.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:26.694134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:26.695343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 614/1222] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:56:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:56:26.700743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906532e-041a-4487-86ba-520c200dfcea None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.798974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906532e-041a-4487-86ba-520c200dfcea None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.824537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:26.835352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfef9d-d1e2-4fd5-95f6-99f177a1d8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.835974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfef9d-d1e2-4fd5-95f6-99f177a1d8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.836255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfef9d-d1e2-4fd5-95f6-99f177a1d8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.836534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfef9d-d1e2-4fd5-95f6-99f177a1d8a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.837761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfef9d-d1e2-4fd5-95f6-99f177a1d8a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:26.838195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfef9d-d1e2-4fd5-95f6-99f177a1d8a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.839555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfef9d-d1e2-4fd5-95f6-99f177a1d8a7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.847262 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 609/1223] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:56:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:26.853900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfa722-e096-4989-ac90-1d35ae0e6868 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.896919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:26.951869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfa722-e096-4989-ac90-1d35ae0e6868 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.984780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e21d8-b9b7-45b9-8407-3dc17f1b5aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.985439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e21d8-b9b7-45b9-8407-3dc17f1b5aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.985717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e21d8-b9b7-45b9-8407-3dc17f1b5aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.985987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e21d8-b9b7-45b9-8407-3dc17f1b5aa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.995834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e21d8-b9b7-45b9-8407-3dc17f1b5aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.997323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e21d8-b9b7-45b9-8407-3dc17f1b5aa6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-b00e21d8-b9b7-45b9-8407-3dc17f1b5aa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:27.012560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:27.013696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:27.014985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:27.014985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:27.014985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:27.014985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:27.014985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 615/1224] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:56:27.019758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfef9d-d1e2-4fd5-95f6-99f177a1d8a7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:27.118814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfef9d-d1e2-4fd5-95f6-99f177a1d8a7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:27.152816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044ec99-857e-4caf-946e-41b2853262c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:27.153450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044ec99-857e-4caf-946e-41b2853262c5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:27.153734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044ec99-857e-4caf-946e-41b2853262c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:27.154014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044ec99-857e-4caf-946e-41b2853262c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:27.154700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044ec99-857e-4caf-946e-41b2853262c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:27.155007 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044ec99-857e-4caf-946e-41b2853262c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:27.156391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044ec99-857e-4caf-946e-41b2853262c5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:27.168051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 610/1225] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:27.173922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e21d8-b9b7-45b9-8407-3dc17f1b5aa6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:27.266984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:27.273608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e21d8-b9b7-45b9-8407-3dc17f1b5aa6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:27.306307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65b12d-0a71-4ba4-954b-3f8888384dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:27.306941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b12d-0a71-4ba4-954b-3f8888384dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:27.307227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b12d-0a71-4ba4-954b-3f8888384dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:27.307503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b12d-0a71-4ba4-954b-3f8888384dd6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:27.314742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65b12d-0a71-4ba4-954b-3f8888384dd6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:27.316196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65b12d-0a71-4ba4-954b-3f8888384dd6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ba65b12d-0a71-4ba4-954b-3f8888384dd6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:56:27.327811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:27.328763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:27.329773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:27.329773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:27.329773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:56:27.329773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:56:27.329773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:56:27.329773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:56:27.329773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:56:27.329773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:56:27.329773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:27.329773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:27.330577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 616/1226] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:27.334787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044ec99-857e-4caf-946e-41b2853262c5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:27.382900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:27.435125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044ec99-857e-4caf-946e-41b2853262c5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:27.468303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83666919-3b50-4a29-9710-83ada42314a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:27.468919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83666919-3b50-4a29-9710-83ada42314a1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:27.469238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83666919-3b50-4a29-9710-83ada42314a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:27.469515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83666919-3b50-4a29-9710-83ada42314a1 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:27.470312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83666919-3b50-4a29-9710-83ada42314a1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:27.470621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83666919-3b50-4a29-9710-83ada42314a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:27.472095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83666919-3b50-4a29-9710-83ada42314a1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:27.488941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 611/1227] 10.222.0.86 () {56 vars in 1143 bytes} [Thu Jul 28 17:56:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:27.495334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65b12d-0a71-4ba4-954b-3f8888384dd6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:27.593561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65b12d-0a71-4ba4-954b-3f8888384dd6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:27.627133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127650a-cdf4-4e5e-9395-d2952d0adebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:27.627852 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127650a-cdf4-4e5e-9395-d2952d0adebc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:27.628090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127650a-cdf4-4e5e-9395-d2952d0adebc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:27.628368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127650a-cdf4-4e5e-9395-d2952d0adebc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:27.640211 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127650a-cdf4-4e5e-9395-d2952d0adebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:27.641709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127650a-cdf4-4e5e-9395-d2952d0adebc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-0127650a-cdf4-4e5e-9395-d2952d0adebc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:27.655632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:27.656657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:27.657964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:27.657964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:27.657964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:27.657964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:27.657964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 617/1228] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:56:27.662528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83666919-3b50-4a29-9710-83ada42314a1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:27.764282 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83666919-3b50-4a29-9710-83ada42314a1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:27.766614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:27.797602 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71aabb-7f0b-4f54-bee4-ef84e81e3aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:27.798368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71aabb-7f0b-4f54-bee4-ef84e81e3aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:27.798711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71aabb-7f0b-4f54-bee4-ef84e81e3aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:27.798983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71aabb-7f0b-4f54-bee4-ef84e81e3aa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:27.799668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71aabb-7f0b-4f54-bee4-ef84e81e3aa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:27.799969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71aabb-7f0b-4f54-bee4-ef84e81e3aa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:27.801490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71aabb-7f0b-4f54-bee4-ef84e81e3aa1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:27.813019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 612/1229] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:27.818734 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127650a-cdf4-4e5e-9395-d2952d0adebc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:27.828144 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:27.916901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127650a-cdf4-4e5e-9395-d2952d0adebc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:27.950227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201a5b0-31ed-4e23-88ec-abd4810b00df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:27.950904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201a5b0-31ed-4e23-88ec-abd4810b00df None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:27.951189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201a5b0-31ed-4e23-88ec-abd4810b00df None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:27.951465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201a5b0-31ed-4e23-88ec-abd4810b00df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:27.957959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201a5b0-31ed-4e23-88ec-abd4810b00df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:27.959387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201a5b0-31ed-4e23-88ec-abd4810b00df None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-1201a5b0-31ed-4e23-88ec-abd4810b00df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:56:27.970636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:27.976742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:27.978210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:56:27.978210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:56:27.978210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:56:27.978210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:56:27.978210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:56:27.978210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:56:27.978210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:27.978210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:27.978210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 618/1230] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:56:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:27.978752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71aabb-7f0b-4f54-bee4-ef84e81e3aa1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:28.078450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71aabb-7f0b-4f54-bee4-ef84e81e3aa1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:28.113274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bd584-037d-489a-a3f4-5c7dd3212525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:28.113898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bd584-037d-489a-a3f4-5c7dd3212525 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:28.114184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bd584-037d-489a-a3f4-5c7dd3212525 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:28.114461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bd584-037d-489a-a3f4-5c7dd3212525 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:28.115209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bd584-037d-489a-a3f4-5c7dd3212525 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:28.115520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bd584-037d-489a-a3f4-5c7dd3212525 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:28.116944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bd584-037d-489a-a3f4-5c7dd3212525 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:28.129511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 613/1231] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:56:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:28.136172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201a5b0-31ed-4e23-88ec-abd4810b00df None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:28.203064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:28.236311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201a5b0-31ed-4e23-88ec-abd4810b00df None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:28.271190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1ea0e-1d0f-4e10-ad56-57654ad00abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:28.271918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1ea0e-1d0f-4e10-ad56-57654ad00abb None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:28.272205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1ea0e-1d0f-4e10-ad56-57654ad00abb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:28.272484 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1ea0e-1d0f-4e10-ad56-57654ad00abb None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/b20564a83953433290aa3fabf884370c/roles/ce68cefcc9b14645bbb5a2b411a23a44` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:28.287878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1ea0e-1d0f-4e10-ad56-57654ad00abb None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=b20564a83953433290aa3fabf884370c, role_id=ce68cefcc9b14645bbb5a2b411a23a44)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:28.289560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1ea0e-1d0f-4e10-ad56-57654ad00abb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:28.438540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 619/1232] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:56:28 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/b20564a83953433290aa3fabf884370c/roles/ce68cefcc9b14645bbb5a2b411a23a44 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:30.446925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634ef57-0c7c-4310-9f73-760963294fae None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:30.447454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634ef57-0c7c-4310-9f73-760963294fae None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:30.447564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634ef57-0c7c-4310-9f73-760963294fae None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:30.448843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 614/1233] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:30.454151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9db11-5b73-493d-b315-a2aeca1a0d02 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:30.454545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9db11-5b73-493d-b315-a2aeca1a0d02 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:30.454737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9db11-5b73-493d-b315-a2aeca1a0d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:30.472534 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4df9db11-5b73-493d-b315-a2aeca1a0d02 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:30.473787 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4df9db11-5b73-493d-b315-a2aeca1a0d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:30.538869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 620/1234] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:56:30.563387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634ef57-0c7c-4310-9f73-760963294fae None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:30.588861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:30.664190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634ef57-0c7c-4310-9f73-760963294fae None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:30.698443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681aafb9-9b39-4846-8047-898fb058b128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:30.699054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aafb9-9b39-4846-8047-898fb058b128 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:30.699338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aafb9-9b39-4846-8047-898fb058b128 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:30.699613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aafb9-9b39-4846-8047-898fb058b128 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:30.709507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681aafb9-9b39-4846-8047-898fb058b128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-681aafb9-9b39-4846-8047-898fb058b128 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:56:30.714778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:30.715871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:30.717089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 615/1235] 10.222.0.86 () {56 vars in 1102 bytes} [Thu Jul 28 17:56:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:30.721825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9db11-5b73-493d-b315-a2aeca1a0d02 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:30.767773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:30.819502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9db11-5b73-493d-b315-a2aeca1a0d02 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:30.855361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847d516-dea2-43c3-bff0-d07a24a559e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:30.856010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847d516-dea2-43c3-bff0-d07a24a559e3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:30.856466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847d516-dea2-43c3-bff0-d07a24a559e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:30.857186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847d516-dea2-43c3-bff0-d07a24a559e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:30.857996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847d516-dea2-43c3-bff0-d07a24a559e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:30.858308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847d516-dea2-43c3-bff0-d07a24a559e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:30.859716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847d516-dea2-43c3-bff0-d07a24a559e3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:30.867883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 621/1236] 10.222.0.86 () {56 vars in 1103 bytes} [Thu Jul 28 17:56:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:30.874601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681aafb9-9b39-4846-8047-898fb058b128 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:30.959399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:30.974972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681aafb9-9b39-4846-8047-898fb058b128 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:31.010135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13033e9-a325-4775-aafd-5968f36efa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:31.010747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13033e9-a325-4775-aafd-5968f36efa40 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:31.011025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13033e9-a325-4775-aafd-5968f36efa40 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:31.011295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13033e9-a325-4775-aafd-5968f36efa40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:31.021040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13033e9-a325-4775-aafd-5968f36efa40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:31.022423 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13033e9-a325-4775-aafd-5968f36efa40 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-d13033e9-a325-4775-aafd-5968f36efa40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:31.036669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:31.037658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:31.038666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:31.038666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:31.038666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:31.038666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:31.038666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 616/1237] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:31.044209 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847d516-dea2-43c3-bff0-d07a24a559e3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:31.145071 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847d516-dea2-43c3-bff0-d07a24a559e3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:31.154376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:31.179263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83f5c1-a108-4e0e-99d9-971f7231e6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:31.179883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83f5c1-a108-4e0e-99d9-971f7231e6bf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:31.180167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83f5c1-a108-4e0e-99d9-971f7231e6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:31.180443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83f5c1-a108-4e0e-99d9-971f7231e6bf None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:31.181169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83f5c1-a108-4e0e-99d9-971f7231e6bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:31.181478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83f5c1-a108-4e0e-99d9-971f7231e6bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:31.182839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83f5c1-a108-4e0e-99d9-971f7231e6bf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:31.194184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 622/1238] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:31.199949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13033e9-a325-4775-aafd-5968f36efa40 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:31.301698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13033e9-a325-4775-aafd-5968f36efa40 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:31.334355 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262f18d-3287-4170-9cc6-43ff7a69cc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:31.334995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262f18d-3287-4170-9cc6-43ff7a69cc24 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:31.335280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262f18d-3287-4170-9cc6-43ff7a69cc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:31.335566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262f18d-3287-4170-9cc6-43ff7a69cc24 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:31.338815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:31.343854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262f18d-3287-4170-9cc6-43ff7a69cc24 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:31.345310 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262f18d-3287-4170-9cc6-43ff7a69cc24 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c262f18d-3287-4170-9cc6-43ff7a69cc24 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:56:31.357015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:31.358003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:31.359048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:31.359048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:31.359048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:56:31.359048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:56:31.359048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:56:31.359048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:56:31.359048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:56:31.359048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:56:31.359048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:31.359048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:31.359048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 617/1239] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:31.363953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83f5c1-a108-4e0e-99d9-971f7231e6bf None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:31.464371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83f5c1-a108-4e0e-99d9-971f7231e6bf None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:31.497991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09630fcd-8042-4643-be8a-9eb0612b85ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:31.498598 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09630fcd-8042-4643-be8a-9eb0612b85ae None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:31.498875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09630fcd-8042-4643-be8a-9eb0612b85ae None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:31.499147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09630fcd-8042-4643-be8a-9eb0612b85ae None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:31.499930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09630fcd-8042-4643-be8a-9eb0612b85ae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:31.500232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09630fcd-8042-4643-be8a-9eb0612b85ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:31.501752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09630fcd-8042-4643-be8a-9eb0612b85ae None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:31.518960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 623/1240] 10.222.0.86 () {56 vars in 1143 bytes} [Thu Jul 28 17:56:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:56:31.525362 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262f18d-3287-4170-9cc6-43ff7a69cc24 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:31.622936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262f18d-3287-4170-9cc6-43ff7a69cc24 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:31.655987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49ef59-9962-4770-a704-590abc289eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:31.656609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49ef59-9962-4770-a704-590abc289eff None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:31.656895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49ef59-9962-4770-a704-590abc289eff None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:31.657207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49ef59-9962-4770-a704-590abc289eff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:31.716912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:31.723395 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49ef59-9962-4770-a704-590abc289eff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:31.724845 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49ef59-9962-4770-a704-590abc289eff None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:31.819695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 618/1241] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:56:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:56:31.826057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09630fcd-8042-4643-be8a-9eb0612b85ae None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:31.853656 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:31.935548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09630fcd-8042-4643-be8a-9eb0612b85ae None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:31.969184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8caff-fc85-48d3-82b4-c0c0997c9415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:31.969816 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8caff-fc85-48d3-82b4-c0c0997c9415 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:31.970095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8caff-fc85-48d3-82b4-c0c0997c9415 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:31.970365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8caff-fc85-48d3-82b4-c0c0997c9415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:31.980246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8caff-fc85-48d3-82b4-c0c0997c9415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:31.981694 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8caff-fc85-48d3-82b4-c0c0997c9415 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-63a8caff-fc85-48d3-82b4-c0c0997c9415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:31.996912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:31.998018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:31.999323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:31.999323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:31.999323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:31.999323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:31.999323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 624/1242] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:32.003924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49ef59-9962-4770-a704-590abc289eff None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:32.096862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:32.102643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49ef59-9962-4770-a704-590abc289eff None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:32.139912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabe7a8-416b-4149-b574-108d8e090e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:32.140537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabe7a8-416b-4149-b574-108d8e090e70 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:32.140819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabe7a8-416b-4149-b574-108d8e090e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:32.141127 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabe7a8-416b-4149-b574-108d8e090e70 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:32.141826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabe7a8-416b-4149-b574-108d8e090e70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:32.142132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabe7a8-416b-4149-b574-108d8e090e70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:32.143508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabe7a8-416b-4149-b574-108d8e090e70 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:32.154899 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 619/1243] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:32.160732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8caff-fc85-48d3-82b4-c0c0997c9415 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:32.260446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:32.281124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8caff-fc85-48d3-82b4-c0c0997c9415 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:32.315004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00971cb-bc37-4e10-9873-29281457e0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:32.315643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00971cb-bc37-4e10-9873-29281457e0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:32.315932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00971cb-bc37-4e10-9873-29281457e0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:32.316209 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00971cb-bc37-4e10-9873-29281457e0e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:32.322686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00971cb-bc37-4e10-9873-29281457e0e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:32.324126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00971cb-bc37-4e10-9873-29281457e0e1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-b00971cb-bc37-4e10-9873-29281457e0e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:56:32.336062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:32.337255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:32.338231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:56:32.338231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:56:32.338231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:56:32.338231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:56:32.338231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:56:32.338231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:56:32.338231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:32.338231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:32.338231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 625/1244] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:56:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:32.343117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabe7a8-416b-4149-b574-108d8e090e70 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:32.443940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabe7a8-416b-4149-b574-108d8e090e70 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:32.476087 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233e4b6-2aaa-4930-ac45-2f36bbaef683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:32.476698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233e4b6-2aaa-4930-ac45-2f36bbaef683 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:32.477001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233e4b6-2aaa-4930-ac45-2f36bbaef683 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:32.477281 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233e4b6-2aaa-4930-ac45-2f36bbaef683 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:32.478549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233e4b6-2aaa-4930-ac45-2f36bbaef683 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:32.478854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233e4b6-2aaa-4930-ac45-2f36bbaef683 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:32.480312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233e4b6-2aaa-4930-ac45-2f36bbaef683 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:32.491939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 620/1245] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:56:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:32.498884 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00971cb-bc37-4e10-9873-29281457e0e1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:32.611174 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00971cb-bc37-4e10-9873-29281457e0e1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:32.644954 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd8c3e-20a8-43ce-8b5f-d3d109230195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:32.645597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd8c3e-20a8-43ce-8b5f-d3d109230195 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:32.645880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd8c3e-20a8-43ce-8b5f-d3d109230195 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:32.646155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd8c3e-20a8-43ce-8b5f-d3d109230195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:32.646974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd8c3e-20a8-43ce-8b5f-d3d109230195 None admin] RBAC: Adding query filter params (role.id=ce68cefcc9b14645bbb5a2b411a23a44, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=b20564a83953433290aa3fabf884370c) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:32.647284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd8c3e-20a8-43ce-8b5f-d3d109230195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:32.648711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd8c3e-20a8-43ce-8b5f-d3d109230195 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:32.654406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 626/1246] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:56:32 2022] GET /identity/v3/role_assignments?user.id=b20564a83953433290aa3fabf884370c&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=ce68cefcc9b14645bbb5a2b411a23a44 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:34.698876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ec4f6-d0e6-486c-891b-282599468f2e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:34.699421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ec4f6-d0e6-486c-891b-282599468f2e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:34.699421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ec4f6-d0e6-486c-891b-282599468f2e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:34.700860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 621/1247] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:34.705974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9373d6-9c88-44e2-ae83-521f339493b9 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:34.706389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9373d6-9c88-44e2-ae83-521f339493b9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:34.706521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9373d6-9c88-44e2-ae83-521f339493b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:34.713263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:34.726017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4d9373d6-9c88-44e2-ae83-521f339493b9 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:34.728057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4d9373d6-9c88-44e2-ae83-521f339493b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:34.797324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 627/1248] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:56:34.821040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ec4f6-d0e6-486c-891b-282599468f2e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:34.840165 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:34.930825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ec4f6-d0e6-486c-891b-282599468f2e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:34.967773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a25b32-6d5b-40b8-bfa0-680c9d66a631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:34.968372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a25b32-6d5b-40b8-bfa0-680c9d66a631 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:34.968654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a25b32-6d5b-40b8-bfa0-680c9d66a631 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:34.968928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a25b32-6d5b-40b8-bfa0-680c9d66a631 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:34.979793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a25b32-6d5b-40b8-bfa0-680c9d66a631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-36a25b32-6d5b-40b8-bfa0-680c9d66a631 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:56:34.984671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:34.985790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:34.987011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 622/1249] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:56:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:34.992097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9373d6-9c88-44e2-ae83-521f339493b9 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.091119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9373d6-9c88-44e2-ae83-521f339493b9 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.119999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:35.124765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407ad9d-28fe-4718-8938-0902a9530524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.125408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407ad9d-28fe-4718-8938-0902a9530524 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.125764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407ad9d-28fe-4718-8938-0902a9530524 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.126055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407ad9d-28fe-4718-8938-0902a9530524 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.126765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407ad9d-28fe-4718-8938-0902a9530524 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:35.127071 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407ad9d-28fe-4718-8938-0902a9530524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.128417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407ad9d-28fe-4718-8938-0902a9530524 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.136533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 628/1250] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:56:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:35.142322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a25b32-6d5b-40b8-bfa0-680c9d66a631 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.219932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:35.241811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a25b32-6d5b-40b8-bfa0-680c9d66a631 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.275856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c4b04-23ab-44b9-809e-ebe9689a6c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.276508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c4b04-23ab-44b9-809e-ebe9689a6c89 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.276785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c4b04-23ab-44b9-809e-ebe9689a6c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.277092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c4b04-23ab-44b9-809e-ebe9689a6c89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.287957 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c4b04-23ab-44b9-809e-ebe9689a6c89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.289434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c4b04-23ab-44b9-809e-ebe9689a6c89 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-6b6c4b04-23ab-44b9-809e-ebe9689a6c89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:35.303772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:35.304843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:35.311402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:35.311402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:35.311402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:35.311402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:35.311402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 623/1251] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:35.311742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407ad9d-28fe-4718-8938-0902a9530524 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.410567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407ad9d-28fe-4718-8938-0902a9530524 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.443891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7240b-d474-4469-a48a-7fa2ab05af4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.444505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7240b-d474-4469-a48a-7fa2ab05af4f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.444784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7240b-d474-4469-a48a-7fa2ab05af4f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.445088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7240b-d474-4469-a48a-7fa2ab05af4f None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.445772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7240b-d474-4469-a48a-7fa2ab05af4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:35.446576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7240b-d474-4469-a48a-7fa2ab05af4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.448011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7240b-d474-4469-a48a-7fa2ab05af4f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.459420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 629/1252] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:35.465868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c4b04-23ab-44b9-809e-ebe9689a6c89 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.564924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c4b04-23ab-44b9-809e-ebe9689a6c89 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.591300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:35.599120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f03cd3-229e-4649-83e1-fffa6d428a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.599752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f03cd3-229e-4649-83e1-fffa6d428a35 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.600037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f03cd3-229e-4649-83e1-fffa6d428a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.600309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f03cd3-229e-4649-83e1-fffa6d428a35 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.607784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f03cd3-229e-4649-83e1-fffa6d428a35 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.609256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f03cd3-229e-4649-83e1-fffa6d428a35 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-d9f03cd3-229e-4649-83e1-fffa6d428a35 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:56:35.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:35.622462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:35.623614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:35.623614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:35.623614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:56:35.623614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:56:35.623614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:56:35.623614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:56:35.623614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:56:35.623614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:56:35.623614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:35.623614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:35.625125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 624/1253] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:56:35.629798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7240b-d474-4469-a48a-7fa2ab05af4f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.684885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:35.728076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7240b-d474-4469-a48a-7fa2ab05af4f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.762577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792a72-1277-4c91-9146-c0d73adca1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.763203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792a72-1277-4c91-9146-c0d73adca1fd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.763489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792a72-1277-4c91-9146-c0d73adca1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.763766 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792a72-1277-4c91-9146-c0d73adca1fd None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.764565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792a72-1277-4c91-9146-c0d73adca1fd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:35.764876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792a72-1277-4c91-9146-c0d73adca1fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.766336 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792a72-1277-4c91-9146-c0d73adca1fd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.783158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 630/1254] 10.222.0.86 () {56 vars in 1143 bytes} [Thu Jul 28 17:56:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:35.788897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f03cd3-229e-4649-83e1-fffa6d428a35 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.888490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f03cd3-229e-4649-83e1-fffa6d428a35 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.920650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ff6b2-9354-4392-a9ed-84120ed8ac47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.921297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ff6b2-9354-4392-a9ed-84120ed8ac47 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.921612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ff6b2-9354-4392-a9ed-84120ed8ac47 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.921890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ff6b2-9354-4392-a9ed-84120ed8ac47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.976150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:35.988137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ff6b2-9354-4392-a9ed-84120ed8ac47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.989648 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ff6b2-9354-4392-a9ed-84120ed8ac47 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:36.087909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 625/1255] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:56:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:56:36.093821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792a72-1277-4c91-9146-c0d73adca1fd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:36.191852 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792a72-1277-4c91-9146-c0d73adca1fd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:36.206405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:36.226786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c270b-e4c9-4902-bbea-3d4983478e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:36.227499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c270b-e4c9-4902-bbea-3d4983478e95 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:36.227781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c270b-e4c9-4902-bbea-3d4983478e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:36.228055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c270b-e4c9-4902-bbea-3d4983478e95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:36.237842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c270b-e4c9-4902-bbea-3d4983478e95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:36.239282 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c270b-e4c9-4902-bbea-3d4983478e95 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-956c270b-e4c9-4902-bbea-3d4983478e95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:36.254303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:36.255369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:36.256546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:36.256546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:36.256546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:36.256546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:36.256546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 631/1256] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:36.262058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ff6b2-9354-4392-a9ed-84120ed8ac47 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:36.356696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:36.369260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ff6b2-9354-4392-a9ed-84120ed8ac47 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:36.402015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b0c6f-dae0-4452-a4b2-6a0ff5c8a489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:36.402661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b0c6f-dae0-4452-a4b2-6a0ff5c8a489 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:36.403059 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b0c6f-dae0-4452-a4b2-6a0ff5c8a489 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:36.403345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b0c6f-dae0-4452-a4b2-6a0ff5c8a489 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:36.404085 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b0c6f-dae0-4452-a4b2-6a0ff5c8a489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:36.404393 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b0c6f-dae0-4452-a4b2-6a0ff5c8a489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:36.405792 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b0c6f-dae0-4452-a4b2-6a0ff5c8a489 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:36.418418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 626/1257] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:36.424011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c270b-e4c9-4902-bbea-3d4983478e95 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:36.523068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c270b-e4c9-4902-bbea-3d4983478e95 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:36.556765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96954e55-1e39-4c62-b385-3081f7864631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:36.564152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96954e55-1e39-4c62-b385-3081f7864631 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:36.564152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96954e55-1e39-4c62-b385-3081f7864631 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:36.564152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96954e55-1e39-4c62-b385-3081f7864631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:36.564493 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96954e55-1e39-4c62-b385-3081f7864631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:36.565953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96954e55-1e39-4c62-b385-3081f7864631 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-96954e55-1e39-4c62-b385-3081f7864631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:56:36.579663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:36.580779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:36.583324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:56:36.583324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:56:36.583324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:56:36.583324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:56:36.583324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:56:36.583324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:56:36.583324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:36.583324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:36.583324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 632/1258] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:36.586949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b0c6f-dae0-4452-a4b2-6a0ff5c8a489 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:36.693100 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b0c6f-dae0-4452-a4b2-6a0ff5c8a489 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:36.727045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31258f-67dd-41c1-8318-d1f7d557cbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:36.728277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31258f-67dd-41c1-8318-d1f7d557cbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:36.728560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31258f-67dd-41c1-8318-d1f7d557cbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:36.728836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31258f-67dd-41c1-8318-d1f7d557cbf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:36.729621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31258f-67dd-41c1-8318-d1f7d557cbf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:36.729934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31258f-67dd-41c1-8318-d1f7d557cbf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:36.731366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31258f-67dd-41c1-8318-d1f7d557cbf7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:36.743177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 627/1259] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:56:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:36.749739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96954e55-1e39-4c62-b385-3081f7864631 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:36.781924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:36.848318 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96954e55-1e39-4c62-b385-3081f7864631 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:36.884469 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6b057-b358-4f43-aab6-556c3cb4166f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:36.885125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6b057-b358-4f43-aab6-556c3cb4166f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:36.885407 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6b057-b358-4f43-aab6-556c3cb4166f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:36.885678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6b057-b358-4f43-aab6-556c3cb4166f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:36.886954 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6b057-b358-4f43-aab6-556c3cb4166f None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=b20564a83953433290aa3fabf884370c) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:36.887519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6b057-b358-4f43-aab6-556c3cb4166f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:36.890188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6b057-b358-4f43-aab6-556c3cb4166f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:36.899054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 633/1260] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:56:36 2022] GET /identity/v3/role_assignments?user.id=b20564a83953433290aa3fabf884370c&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:38.916169 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbd2cb-92d3-42c4-90e4-6013a5df25a6 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:38.916721 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbd2cb-92d3-42c4-90e4-6013a5df25a6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:38.916721 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbd2cb-92d3-42c4-90e4-6013a5df25a6 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:38.918110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 628/1261] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:38.923370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087644c-f7ea-40d1-9211-2b114457d0ad None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:38.927904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087644c-f7ea-40d1-9211-2b114457d0ad None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:38.927904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087644c-f7ea-40d1-9211-2b114457d0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:38.942129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3087644c-f7ea-40d1-9211-2b114457d0ad None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:38.943326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3087644c-f7ea-40d1-9211-2b114457d0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:39.009698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 634/1262] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:56:39.031607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbd2cb-92d3-42c4-90e4-6013a5df25a6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.042379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:39.129563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbd2cb-92d3-42c4-90e4-6013a5df25a6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.163433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890403d-f747-4c3f-851d-9e70b37e1ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.164065 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890403d-f747-4c3f-851d-9e70b37e1ecb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.164349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890403d-f747-4c3f-851d-9e70b37e1ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.164650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890403d-f747-4c3f-851d-9e70b37e1ecb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.180054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890403d-f747-4c3f-851d-9e70b37e1ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-f890403d-f747-4c3f-851d-9e70b37e1ecb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:56:39.181247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:39.182334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:39.183406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 629/1263] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:56:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:39.188385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087644c-f7ea-40d1-9211-2b114457d0ad None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.207333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:39.286660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087644c-f7ea-40d1-9211-2b114457d0ad None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.322260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff3b7a-4b3d-4348-8a60-9e014b805aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.322883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff3b7a-4b3d-4348-8a60-9e014b805aad None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.323166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff3b7a-4b3d-4348-8a60-9e014b805aad None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.323443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff3b7a-4b3d-4348-8a60-9e014b805aad None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.324163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff3b7a-4b3d-4348-8a60-9e014b805aad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:39.324470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff3b7a-4b3d-4348-8a60-9e014b805aad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.325857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff3b7a-4b3d-4348-8a60-9e014b805aad None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.334058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 635/1264] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:56:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:39.339868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890403d-f747-4c3f-851d-9e70b37e1ecb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.406080 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:39.440084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890403d-f747-4c3f-851d-9e70b37e1ecb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.472958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8994c5b-39a0-4a5a-8853-fff772ffca5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.473623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8994c5b-39a0-4a5a-8853-fff772ffca5e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.473904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8994c5b-39a0-4a5a-8853-fff772ffca5e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.474177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8994c5b-39a0-4a5a-8853-fff772ffca5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.484870 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8994c5b-39a0-4a5a-8853-fff772ffca5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.486375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8994c5b-39a0-4a5a-8853-fff772ffca5e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-f8994c5b-39a0-4a5a-8853-fff772ffca5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:39.501022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:39.501967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:39.503006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:39.503006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:39.503006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:39.503006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:39.503006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 630/1265] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:39.508141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff3b7a-4b3d-4348-8a60-9e014b805aad None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.593280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:39.613975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff3b7a-4b3d-4348-8a60-9e014b805aad None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.649911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1128f9e-0760-49d9-942d-fc514713ddae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.650601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1128f9e-0760-49d9-942d-fc514713ddae None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.650861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1128f9e-0760-49d9-942d-fc514713ddae None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.651132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1128f9e-0760-49d9-942d-fc514713ddae None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.651824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1128f9e-0760-49d9-942d-fc514713ddae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:39.652124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1128f9e-0760-49d9-942d-fc514713ddae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.653593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1128f9e-0760-49d9-942d-fc514713ddae None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.665859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 636/1266] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:39.671867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8994c5b-39a0-4a5a-8853-fff772ffca5e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.771251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8994c5b-39a0-4a5a-8853-fff772ffca5e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.791379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:39.804733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348016b0-8095-4477-9c45-9abeb97a9e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.805428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348016b0-8095-4477-9c45-9abeb97a9e26 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.805711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348016b0-8095-4477-9c45-9abeb97a9e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.805987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348016b0-8095-4477-9c45-9abeb97a9e26 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.813236 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348016b0-8095-4477-9c45-9abeb97a9e26 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.814679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348016b0-8095-4477-9c45-9abeb97a9e26 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-348016b0-8095-4477-9c45-9abeb97a9e26 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:56:39.825685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:39.826851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:39.828016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:39.828016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:39.828016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:56:39.828016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:56:39.828016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:56:39.828016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:56:39.828016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:56:39.828016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:56:39.828016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:39.828016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:39.828016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 631/1267] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:39.832916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1128f9e-0760-49d9-942d-fc514713ddae None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.937412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1128f9e-0760-49d9-942d-fc514713ddae None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.976438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e096ba5-4f6c-4dec-9864-7e42ad8db693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.977714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e096ba5-4f6c-4dec-9864-7e42ad8db693 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.978029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e096ba5-4f6c-4dec-9864-7e42ad8db693 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.978330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e096ba5-4f6c-4dec-9864-7e42ad8db693 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.979126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e096ba5-4f6c-4dec-9864-7e42ad8db693 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:39.979436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e096ba5-4f6c-4dec-9864-7e42ad8db693 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.980862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e096ba5-4f6c-4dec-9864-7e42ad8db693 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.990618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:40.000646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 637/1268] 10.222.0.86 () {56 vars in 1143 bytes} [Thu Jul 28 17:56:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:40.006616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348016b0-8095-4477-9c45-9abeb97a9e26 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:40.106019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348016b0-8095-4477-9c45-9abeb97a9e26 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:40.139561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaddf0-cb78-489f-9686-e0089e3e2723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:40.140201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaddf0-cb78-489f-9686-e0089e3e2723 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:40.140485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaddf0-cb78-489f-9686-e0089e3e2723 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:40.140764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaddf0-cb78-489f-9686-e0089e3e2723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:40.151732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaaddf0-cb78-489f-9686-e0089e3e2723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:40.153212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaaddf0-cb78-489f-9686-e0089e3e2723 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3aaaddf0-cb78-489f-9686-e0089e3e2723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:40.167962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:40.173561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:40.174580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:40.174580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:40.174580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:40.174580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:40.174580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 632/1269] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:40.176212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e096ba5-4f6c-4dec-9864-7e42ad8db693 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:40.286958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e096ba5-4f6c-4dec-9864-7e42ad8db693 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:40.321716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb60f2-c676-4e2c-b196-06769dd25467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:40.322334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb60f2-c676-4e2c-b196-06769dd25467 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:40.322617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb60f2-c676-4e2c-b196-06769dd25467 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:40.322891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb60f2-c676-4e2c-b196-06769dd25467 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:40.323582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb60f2-c676-4e2c-b196-06769dd25467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:40.323892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb60f2-c676-4e2c-b196-06769dd25467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:40.325279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb60f2-c676-4e2c-b196-06769dd25467 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:40.337012 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 638/1270] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:40.342795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaddf0-cb78-489f-9686-e0089e3e2723 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:40.379857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:40.447603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaddf0-cb78-489f-9686-e0089e3e2723 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:40.480307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e49192-81f4-4893-962b-6b33619dd1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:40.480945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e49192-81f4-4893-962b-6b33619dd1ff None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:40.481265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e49192-81f4-4893-962b-6b33619dd1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:40.481544 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e49192-81f4-4893-962b-6b33619dd1ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:40.487974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e49192-81f4-4893-962b-6b33619dd1ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:40.493044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e49192-81f4-4893-962b-6b33619dd1ff None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-23e49192-81f4-4893-962b-6b33619dd1ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:56:40.501326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:40.502373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:40.503475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:56:40.503475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:56:40.503475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:56:40.503475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:56:40.503475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:56:40.503475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:56:40.503475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:40.503475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:40.503475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 633/1271] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:56:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:40.509070 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb60f2-c676-4e2c-b196-06769dd25467 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:40.569634 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:40.609995 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb60f2-c676-4e2c-b196-06769dd25467 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:40.647683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febaeb1-17f3-43b5-ade2-76fca9f8e89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:40.647683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febaeb1-17f3-43b5-ade2-76fca9f8e89f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:40.647683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febaeb1-17f3-43b5-ade2-76fca9f8e89f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:40.647683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febaeb1-17f3-43b5-ade2-76fca9f8e89f None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:40.648278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febaeb1-17f3-43b5-ade2-76fca9f8e89f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:40.648278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febaeb1-17f3-43b5-ade2-76fca9f8e89f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:40.649653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febaeb1-17f3-43b5-ade2-76fca9f8e89f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:40.662118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 639/1272] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:56:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:40.668348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e49192-81f4-4893-962b-6b33619dd1ff None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:40.762270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:40.770793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e49192-81f4-4893-962b-6b33619dd1ff None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:40.805768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fe03b-35f8-462f-9e25-5724f9b55321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:40.806440 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fe03b-35f8-462f-9e25-5724f9b55321 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:40.806720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fe03b-35f8-462f-9e25-5724f9b55321 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:40.807052 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fe03b-35f8-462f-9e25-5724f9b55321 None admin] PATH_INFO: `/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/b20564a83953433290aa3fabf884370c/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:40.809341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3fe03b-35f8-462f-9e25-5724f9b55321 None admin] RBAC: Authorizing `identity:create_grant(project_id=bae648a61bc8421a947f6f4c30413b5d, user_id=b20564a83953433290aa3fabf884370c, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:40.810946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3fe03b-35f8-462f-9e25-5724f9b55321 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:40.911854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 634/1273] 10.222.0.86 () {58 vars in 1439 bytes} [Thu Jul 28 17:56:40 2022] PUT /identity/v3/projects/bae648a61bc8421a947f6f4c30413b5d/users/b20564a83953433290aa3fabf884370c/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:42.890210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54df3fe-08f8-49b8-96aa-e085b5e47e94 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:42.890767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54df3fe-08f8-49b8-96aa-e085b5e47e94 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:42.890767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54df3fe-08f8-49b8-96aa-e085b5e47e94 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:42.892142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 640/1274] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:42.897222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66b73f-7121-4d67-a636-5a9c0230ad28 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:42.897656 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66b73f-7121-4d67-a636-5a9c0230ad28 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:42.897785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66b73f-7121-4d67-a636-5a9c0230ad28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:42.916863 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-2a66b73f-7121-4d67-a636-5a9c0230ad28 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:42.918090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2a66b73f-7121-4d67-a636-5a9c0230ad28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:42.984268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 635/1275] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:56:43.007674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54df3fe-08f8-49b8-96aa-e085b5e47e94 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:43.108935 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54df3fe-08f8-49b8-96aa-e085b5e47e94 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:43.126467 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:43.152085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfe7e7-b2cb-4aa2-bfef-08add105ebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:43.152085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfe7e7-b2cb-4aa2-bfef-08add105ebe1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:43.152085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfe7e7-b2cb-4aa2-bfef-08add105ebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:43.152085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfe7e7-b2cb-4aa2-bfef-08add105ebe1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:43.154586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cfe7e7-b2cb-4aa2-bfef-08add105ebe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-68cfe7e7-b2cb-4aa2-bfef-08add105ebe1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:56:43.159549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:43.160760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:43.161803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 641/1276] 10.222.0.86 () {56 vars in 1096 bytes} [Thu Jul 28 17:56:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:43.166938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66b73f-7121-4d67-a636-5a9c0230ad28 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:43.247118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:43.265799 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66b73f-7121-4d67-a636-5a9c0230ad28 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:43.300534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efefdbd-71b8-40af-8a87-cb9d40353d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:43.301185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efefdbd-71b8-40af-8a87-cb9d40353d18 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:43.301473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efefdbd-71b8-40af-8a87-cb9d40353d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:43.301757 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efefdbd-71b8-40af-8a87-cb9d40353d18 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:43.302481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efefdbd-71b8-40af-8a87-cb9d40353d18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:43.302796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efefdbd-71b8-40af-8a87-cb9d40353d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:43.304166 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efefdbd-71b8-40af-8a87-cb9d40353d18 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:43.312444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 636/1277] 10.222.0.86 () {56 vars in 1099 bytes} [Thu Jul 28 17:56:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:43.318323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfe7e7-b2cb-4aa2-bfef-08add105ebe1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:43.422747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfe7e7-b2cb-4aa2-bfef-08add105ebe1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:43.456672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81969c-923c-4505-80f9-7f729d7a9712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:43.457345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81969c-923c-4505-80f9-7f729d7a9712 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:43.457631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81969c-923c-4505-80f9-7f729d7a9712 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:43.457908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81969c-923c-4505-80f9-7f729d7a9712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:43.468813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81969c-923c-4505-80f9-7f729d7a9712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:43.470335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81969c-923c-4505-80f9-7f729d7a9712 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-eb81969c-923c-4505-80f9-7f729d7a9712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:43.484591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:43.485710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:43.486975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:43.486975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:43.486975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:43.486975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:43.486975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 642/1278] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:43.491613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efefdbd-71b8-40af-8a87-cb9d40353d18 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:43.590993 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efefdbd-71b8-40af-8a87-cb9d40353d18 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:43.624804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729c829-4f43-4fec-bd9c-011e1beac4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:43.625450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729c829-4f43-4fec-bd9c-011e1beac4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:43.625733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729c829-4f43-4fec-bd9c-011e1beac4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:43.626009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729c829-4f43-4fec-bd9c-011e1beac4b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:43.626699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729c829-4f43-4fec-bd9c-011e1beac4b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:43.627006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729c829-4f43-4fec-bd9c-011e1beac4b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:43.628369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729c829-4f43-4fec-bd9c-011e1beac4b7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:43.640417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 637/1279] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:43.646349 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81969c-923c-4505-80f9-7f729d7a9712 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:43.678224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:43.747188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81969c-923c-4505-80f9-7f729d7a9712 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:43.780065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522216fe-f8cb-4257-b5c8-fa96b6559aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:43.780703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522216fe-f8cb-4257-b5c8-fa96b6559aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:43.781005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522216fe-f8cb-4257-b5c8-fa96b6559aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:43.781284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522216fe-f8cb-4257-b5c8-fa96b6559aa7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:43.788645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522216fe-f8cb-4257-b5c8-fa96b6559aa7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:43.790714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522216fe-f8cb-4257-b5c8-fa96b6559aa7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-522216fe-f8cb-4257-b5c8-fa96b6559aa7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:56:43.801877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:43.803119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:43.804264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:43.804264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:43.804264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:56:43.804264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:56:43.804264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:56:43.804264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:56:43.804264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:56:43.804264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:56:43.804264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:56:43.804264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:43.804264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 643/1280] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:43.808950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729c829-4f43-4fec-bd9c-011e1beac4b7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:43.812802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:43.907618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729c829-4f43-4fec-bd9c-011e1beac4b7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:43.941913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb0ade-1b13-4415-b777-bc4e41ba74fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:43.942459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb0ade-1b13-4415-b777-bc4e41ba74fe None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:43.942739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb0ade-1b13-4415-b777-bc4e41ba74fe None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:43.943014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb0ade-1b13-4415-b777-bc4e41ba74fe None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:43.943800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb0ade-1b13-4415-b777-bc4e41ba74fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:43.944103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb0ade-1b13-4415-b777-bc4e41ba74fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:43.946096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb0ade-1b13-4415-b777-bc4e41ba74fe None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:43.963110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 638/1281] 10.222.0.86 () {56 vars in 1143 bytes} [Thu Jul 28 17:56:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:43.969097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522216fe-f8cb-4257-b5c8-fa96b6559aa7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:44.058807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:44.068057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522216fe-f8cb-4257-b5c8-fa96b6559aa7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:44.101687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae60203-45f3-4dea-adff-d880d1aefe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:44.102329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60203-45f3-4dea-adff-d880d1aefe25 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:44.102617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60203-45f3-4dea-adff-d880d1aefe25 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:44.102884 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60203-45f3-4dea-adff-d880d1aefe25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:44.171631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae60203-45f3-4dea-adff-d880d1aefe25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:44.173197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae60203-45f3-4dea-adff-d880d1aefe25 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:44.263246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:44.273359 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 644/1282] 10.222.0.86 () {58 vars in 1282 bytes} [Thu Jul 28 17:56:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:56:44.280166 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb0ade-1b13-4415-b777-bc4e41ba74fe None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:44.341631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:44.382337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb0ade-1b13-4415-b777-bc4e41ba74fe None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:44.421050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45587b-ebad-432b-855e-b7f36a3e9791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:44.421713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45587b-ebad-432b-855e-b7f36a3e9791 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:44.422000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45587b-ebad-432b-855e-b7f36a3e9791 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:44.422277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45587b-ebad-432b-855e-b7f36a3e9791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:44.432564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45587b-ebad-432b-855e-b7f36a3e9791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:44.434054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45587b-ebad-432b-855e-b7f36a3e9791 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ce45587b-ebad-432b-855e-b7f36a3e9791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:56:44.448921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:44.449997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:44.450966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:56:44.450966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:56:44.450966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:56:44.450966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:44.450966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 639/1283] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 17:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:44.455942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae60203-45f3-4dea-adff-d880d1aefe25 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:44.554519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae60203-45f3-4dea-adff-d880d1aefe25 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:44.588296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ceb69-ce26-4387-872f-48fd119da705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:44.588904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ceb69-ce26-4387-872f-48fd119da705 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:44.589213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ceb69-ce26-4387-872f-48fd119da705 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:44.589489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ceb69-ce26-4387-872f-48fd119da705 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:44.590212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ceb69-ce26-4387-872f-48fd119da705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:44.590517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ceb69-ce26-4387-872f-48fd119da705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:44.591910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ceb69-ce26-4387-872f-48fd119da705 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:44.603206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 645/1284] 10.222.0.86 () {56 vars in 1109 bytes} [Thu Jul 28 17:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:44.609022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45587b-ebad-432b-855e-b7f36a3e9791 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:44.713468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45587b-ebad-432b-855e-b7f36a3e9791 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:44.727107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:44.747280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb943f5f-6f69-47e3-b7d3-378e50e7f384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:44.747921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb943f5f-6f69-47e3-b7d3-378e50e7f384 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:44.748203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb943f5f-6f69-47e3-b7d3-378e50e7f384 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:44.748480 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb943f5f-6f69-47e3-b7d3-378e50e7f384 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:44.755156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb943f5f-6f69-47e3-b7d3-378e50e7f384 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:44.756593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb943f5f-6f69-47e3-b7d3-378e50e7f384 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-eb943f5f-6f69-47e3-b7d3-378e50e7f384 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:56:44.768268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:44.769386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:44.770424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:56:44.770424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:56:44.770424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:56:44.770424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:56:44.770424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:56:44.770424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:56:44.770424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:56:44.770424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Jul 28 17:56:44.770424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 640/1285] 10.222.0.86 () {56 vars in 1111 bytes} [Thu Jul 28 17:56:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:44.775393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ceb69-ce26-4387-872f-48fd119da705 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:44.825476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:44.873489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ceb69-ce26-4387-872f-48fd119da705 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:44.905896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066dd72e-7a29-4cb1-82dc-64b8f0221b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:44.906508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dd72e-7a29-4cb1-82dc-64b8f0221b71 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:44.906790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dd72e-7a29-4cb1-82dc-64b8f0221b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:44.907063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dd72e-7a29-4cb1-82dc-64b8f0221b71 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:44.907807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066dd72e-7a29-4cb1-82dc-64b8f0221b71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:44.908115 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066dd72e-7a29-4cb1-82dc-64b8f0221b71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:44.909569 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066dd72e-7a29-4cb1-82dc-64b8f0221b71 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:44.922519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 646/1286] 10.222.0.86 () {56 vars in 1148 bytes} [Thu Jul 28 17:56:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:44.928619 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb943f5f-6f69-47e3-b7d3-378e50e7f384 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:45.028866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb943f5f-6f69-47e3-b7d3-378e50e7f384 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:45.064956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edb43f-1146-4a3b-a8bf-7afce8e98b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:45.065593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edb43f-1146-4a3b-a8bf-7afce8e98b18 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:45.065957 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edb43f-1146-4a3b-a8bf-7afce8e98b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:45.066286 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edb43f-1146-4a3b-a8bf-7afce8e98b18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:45.067075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edb43f-1146-4a3b-a8bf-7afce8e98b18 None admin] RBAC: Adding query filter params (role.id=61306055d76a4d4695ad0489edb99ba0, scope.project.id=bae648a61bc8421a947f6f4c30413b5d, user.id=b20564a83953433290aa3fabf884370c) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:45.067386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edb43f-1146-4a3b-a8bf-7afce8e98b18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:45.069225 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edb43f-1146-4a3b-a8bf-7afce8e98b18 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:45.075378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 641/1287] 10.222.0.86 () {56 vars in 1374 bytes} [Thu Jul 28 17:56:44 2022] GET /identity/v3/role_assignments?user.id=b20564a83953433290aa3fabf884370c&scope.project.id=bae648a61bc8421a947f6f4c30413b5d&role.id=61306055d76a4d4695ad0489edb99ba0 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:47.110014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62965a9f-3b68-4949-bcab-cf509b97d86e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:47.110611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62965a9f-3b68-4949-bcab-cf509b97d86e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:47.110611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62965a9f-3b68-4949-bcab-cf509b97d86e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:47.111944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 647/1288] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:47.118027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59500d9d-c46e-4721-90b4-4c0515d2fcc1 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:47.118780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59500d9d-c46e-4721-90b4-4c0515d2fcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:47.118892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59500d9d-c46e-4721-90b4-4c0515d2fcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:47.139772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-59500d9d-c46e-4721-90b4-4c0515d2fcc1 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:47.145199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-59500d9d-c46e-4721-90b4-4c0515d2fcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:47.173737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:47.214283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 642/1289] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:56:47.237016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62965a9f-3b68-4949-bcab-cf509b97d86e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:47.348085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62965a9f-3b68-4949-bcab-cf509b97d86e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:47.364719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:47.386207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ede314-aa94-4050-9b42-422632b1a23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:47.386860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ede314-aa94-4050-9b42-422632b1a23e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:47.387146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ede314-aa94-4050-9b42-422632b1a23e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:47.387427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ede314-aa94-4050-9b42-422632b1a23e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:47.388094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ede314-aa94-4050-9b42-422632b1a23e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:47.389523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ede314-aa94-4050-9b42-422632b1a23e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-81ede314-aa94-4050-9b42-422632b1a23e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:47.398562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:47.406235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:47.407304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 648/1290] 10.222.0.86 () {56 vars in 1117 bytes} [Thu Jul 28 17:56:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:56:47.408403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59500d9d-c46e-4721-90b4-4c0515d2fcc1 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:47.516216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59500d9d-c46e-4721-90b4-4c0515d2fcc1 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:47.550279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4931ec-fdb0-44e0-96a2-b6ec0597b451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:47.550900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4931ec-fdb0-44e0-96a2-b6ec0597b451 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:47.551183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4931ec-fdb0-44e0-96a2-b6ec0597b451 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:47.551516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4931ec-fdb0-44e0-96a2-b6ec0597b451 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:47.552277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4931ec-fdb0-44e0-96a2-b6ec0597b451 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:47.552581 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4931ec-fdb0-44e0-96a2-b6ec0597b451 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:47.553998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4931ec-fdb0-44e0-96a2-b6ec0597b451 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:47.559391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 643/1291] 10.222.0.86 () {56 vars in 1116 bytes} [Thu Jul 28 17:56:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:47.565259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ede314-aa94-4050-9b42-422632b1a23e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:47.663704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ede314-aa94-4050-9b42-422632b1a23e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:47.698322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fb7e7-5b28-41c2-9d3a-a30eb84ea001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:47.699018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fb7e7-5b28-41c2-9d3a-a30eb84ea001 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:47.699305 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fb7e7-5b28-41c2-9d3a-a30eb84ea001 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:47.699582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fb7e7-5b28-41c2-9d3a-a30eb84ea001 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:47.700269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fb7e7-5b28-41c2-9d3a-a30eb84ea001 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:47.700611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fb7e7-5b28-41c2-9d3a-a30eb84ea001 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:47.702028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fb7e7-5b28-41c2-9d3a-a30eb84ea001 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:47.706742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 649/1292] 10.222.0.86 () {56 vars in 1116 bytes} [Thu Jul 28 17:56:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:49.716550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ba2d6-f83d-4fe1-9c11-4863d4ddbab3 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:49.717080 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ba2d6-f83d-4fe1-9c11-4863d4ddbab3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:49.717080 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ba2d6-f83d-4fe1-9c11-4863d4ddbab3 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:49.718405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 644/1293] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:49.723597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff65ad-e3b9-4f5f-a74c-a30fcaf20da8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:49.724021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff65ad-e3b9-4f5f-a74c-a30fcaf20da8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:49.724158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff65ad-e3b9-4f5f-a74c-a30fcaf20da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:49.743059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-63ff65ad-e3b9-4f5f-a74c-a30fcaf20da8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:49.744579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-63ff65ad-e3b9-4f5f-a74c-a30fcaf20da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:49.773885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:49.810252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 650/1294] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:56:49.832554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ba2d6-f83d-4fe1-9c11-4863d4ddbab3 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:49.863504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:49.933759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ba2d6-f83d-4fe1-9c11-4863d4ddbab3 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:49.967249 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd621540-b753-4db8-a6a0-35155303851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:49.967800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd621540-b753-4db8-a6a0-35155303851d None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:49.968085 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd621540-b753-4db8-a6a0-35155303851d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:49.968363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd621540-b753-4db8-a6a0-35155303851d None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:49.969128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd621540-b753-4db8-a6a0-35155303851d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:49.970528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd621540-b753-4db8-a6a0-35155303851d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:50.055535 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 645/1295] 10.222.0.86 () {60 vars in 1141 bytes} [Thu Jul 28 17:56:49 2022] POST /identity/v3/services => generated 246 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:52.076813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cd4ad-2858-4463-b4eb-4b1128388bd4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:52.077417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cd4ad-2858-4463-b4eb-4b1128388bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:52.077417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cd4ad-2858-4463-b4eb-4b1128388bd4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:52.079283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 651/1296] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:52.085077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f3636-f754-4d48-84f0-362577638db0 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:52.085515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f3636-f754-4d48-84f0-362577638db0 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:52.085515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f3636-f754-4d48-84f0-362577638db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:52.105091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8c3f3636-f754-4d48-84f0-362577638db0 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:52.106391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8c3f3636-f754-4d48-84f0-362577638db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:52.178838 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 646/1297] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:52 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:56:52.202691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cd4ad-2858-4463-b4eb-4b1128388bd4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:52.303175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cd4ad-2858-4463-b4eb-4b1128388bd4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:52.338141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c3c1f-7aee-4f5f-aa0e-e8ee7c254ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:52.338775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c3c1f-7aee-4f5f-aa0e-e8ee7c254ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:52.339061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c3c1f-7aee-4f5f-aa0e-e8ee7c254ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:52.339336 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c3c1f-7aee-4f5f-aa0e-e8ee7c254ee0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:52.339991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c3c1f-7aee-4f5f-aa0e-e8ee7c254ee0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:52.341391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c3c1f-7aee-4f5f-aa0e-e8ee7c254ee0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-3f2c3c1f-7aee-4f5f-aa0e-e8ee7c254ee0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:52.350781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:52.356539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:52.357709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 652/1298] 10.222.0.86 () {56 vars in 1117 bytes} [Thu Jul 28 17:56:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:52.358719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f3636-f754-4d48-84f0-362577638db0 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:52.378226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:52.468214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f3636-f754-4d48-84f0-362577638db0 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:52.502443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bcbda-c88e-41ed-81da-78cd5ddb9ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:52.503064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bcbda-c88e-41ed-81da-78cd5ddb9ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:52.503346 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bcbda-c88e-41ed-81da-78cd5ddb9ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:52.503621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bcbda-c88e-41ed-81da-78cd5ddb9ff5 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:52.504470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bcbda-c88e-41ed-81da-78cd5ddb9ff5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:52.504778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bcbda-c88e-41ed-81da-78cd5ddb9ff5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:52.506178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bcbda-c88e-41ed-81da-78cd5ddb9ff5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:52.511316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 647/1299] 10.222.0.86 () {56 vars in 1116 bytes} [Thu Jul 28 17:56:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:52.517431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c3c1f-7aee-4f5f-aa0e-e8ee7c254ee0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:52.549682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:52.614952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c3c1f-7aee-4f5f-aa0e-e8ee7c254ee0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:52.647460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff607b9-33b8-4dc2-bb9e-1ffce7e08ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:52.648073 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff607b9-33b8-4dc2-bb9e-1ffce7e08ddf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:52.648915 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff607b9-33b8-4dc2-bb9e-1ffce7e08ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:52.649229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff607b9-33b8-4dc2-bb9e-1ffce7e08ddf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:52.649982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff607b9-33b8-4dc2-bb9e-1ffce7e08ddf None admin] RBAC: Adding query filter params (interface=public, service_id=a3a43e99c0354496bbb9b482f6ad1657, region_id=RegionOne) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:52.650291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff607b9-33b8-4dc2-bb9e-1ffce7e08ddf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:52.651676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff607b9-33b8-4dc2-bb9e-1ffce7e08ddf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:52.656614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 653/1300] 10.222.0.86 () {56 vars in 1251 bytes} [Thu Jul 28 17:56:52 2022] GET /identity/v3/endpoints?service_id=a3a43e99c0354496bbb9b482f6ad1657&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:52.662575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bcbda-c88e-41ed-81da-78cd5ddb9ff5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:52.750269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:52.768807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bcbda-c88e-41ed-81da-78cd5ddb9ff5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:52.802099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16be321-fd0e-4be2-94d6-cf595b59123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:52.802711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16be321-fd0e-4be2-94d6-cf595b59123a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:52.802989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16be321-fd0e-4be2-94d6-cf595b59123a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:52.803364 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16be321-fd0e-4be2-94d6-cf595b59123a None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:52.804012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16be321-fd0e-4be2-94d6-cf595b59123a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:52.805534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16be321-fd0e-4be2-94d6-cf595b59123a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:52.812211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 648/1301] 10.222.0.86 () {56 vars in 1087 bytes} [Thu Jul 28 17:56:52 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:54.803688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a8a5b-d17e-49a3-b990-d4593d6ec6e0 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:54.804241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a8a5b-d17e-49a3-b990-d4593d6ec6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:54.804241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a8a5b-d17e-49a3-b990-d4593d6ec6e0 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:54.805679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 654/1302] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 17:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:56:54.810818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3870921-c851-4c04-8c9b-699c5aa9760f None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:54.811217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3870921-c851-4c04-8c9b-699c5aa9760f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:54.811434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3870921-c851-4c04-8c9b-699c5aa9760f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:54.831658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f3870921-c851-4c04-8c9b-699c5aa9760f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:54.833565 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f3870921-c851-4c04-8c9b-699c5aa9760f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:54.901909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 649/1303] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 17:56:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:56:54.924813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a8a5b-d17e-49a3-b990-d4593d6ec6e0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:55.019573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:55.026857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a8a5b-d17e-49a3-b990-d4593d6ec6e0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:55.060437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fd66e-97e0-4e86-8ecc-63ecab3a307c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:55.061497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd66e-97e0-4e86-8ecc-63ecab3a307c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:55.062301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd66e-97e0-4e86-8ecc-63ecab3a307c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:55.062441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd66e-97e0-4e86-8ecc-63ecab3a307c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:55.062979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518fd66e-97e0-4e86-8ecc-63ecab3a307c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:55.064918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518fd66e-97e0-4e86-8ecc-63ecab3a307c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-518fd66e-97e0-4e86-8ecc-63ecab3a307c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:56:55.082278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:56:55.083601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 28 17:56:55.084487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 655/1304] 10.222.0.86 () {56 vars in 1117 bytes} [Thu Jul 28 17:56:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:56:55.090259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3870921-c851-4c04-8c9b-699c5aa9760f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:55.171369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:55.193473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3870921-c851-4c04-8c9b-699c5aa9760f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:55.226561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48eb14b-32cc-436a-a7df-07e2dff97427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:55.227177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eb14b-32cc-436a-a7df-07e2dff97427 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:55.227459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eb14b-32cc-436a-a7df-07e2dff97427 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:55.227736 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eb14b-32cc-436a-a7df-07e2dff97427 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:55.228419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48eb14b-32cc-436a-a7df-07e2dff97427 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:55.228727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48eb14b-32cc-436a-a7df-07e2dff97427 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:55.230142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48eb14b-32cc-436a-a7df-07e2dff97427 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:55.235837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 650/1305] 10.222.0.86 () {56 vars in 1116 bytes} [Thu Jul 28 17:56:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:55.242415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fd66e-97e0-4e86-8ecc-63ecab3a307c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:55.342394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fd66e-97e0-4e86-8ecc-63ecab3a307c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:55.375592 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958209d-73f9-481d-9fa2-35570361ea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:55.376200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958209d-73f9-481d-9fa2-35570361ea2e None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:55.376482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958209d-73f9-481d-9fa2-35570361ea2e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:55.376756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958209d-73f9-481d-9fa2-35570361ea2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:55.377487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958209d-73f9-481d-9fa2-35570361ea2e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:55.378860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958209d-73f9-481d-9fa2-35570361ea2e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:55.495518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 656/1306] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:55 2022] POST /identity/v3/endpoints => generated 331 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:57:05.233189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=30 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:57:05.489740 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:47.272016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fcb9b-804e-4185-9197-a6998275459f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:47.273056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fcb9b-804e-4185-9197-a6998275459f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:47.273574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fcb9b-804e-4185-9197-a6998275459f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:47.275208 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 651/1307] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:03:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:03:47.280706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f26963-db0e-4809-8c2d-54882f1461cb None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:47.281578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f26963-db0e-4809-8c2d-54882f1461cb None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:47.282076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f26963-db0e-4809-8c2d-54882f1461cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:47.361024 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a6f26963-db0e-4809-8c2d-54882f1461cb None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:47.362511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a6f26963-db0e-4809-8c2d-54882f1461cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:47.505655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 657/1308] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:47.570679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b6595-b65d-4f6e-bcae-133a974af576 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:47.571326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b6595-b65d-4f6e-bcae-133a974af576 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:47.572029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b6595-b65d-4f6e-bcae-133a974af576 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:47.574819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 652/1309] 10.222.0.86 () {54 vars in 965 bytes} [Thu Jul 28 18:03:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:03:47.581697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0c81-d78f-4f98-9293-33eb4e0da7e0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:47.584327 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0c81-d78f-4f98-9293-33eb4e0da7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:47.584327 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0c81-d78f-4f98-9293-33eb4e0da7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:47.691741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ea6a0c81-d78f-4f98-9293-33eb4e0da7e0 None None] MFA Rules not processed for user `b038336582a0435dbd47ad2c09cd23e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:47.691741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ea6a0c81-d78f-4f98-9293-33eb4e0da7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:47.710356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:47.797680 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 658/1310] 10.222.0.86 () {58 vars in 1062 bytes} [Thu Jul 28 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:47.806483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b6595-b65d-4f6e-bcae-133a974af576 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:47.808464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-825b6595-b65d-4f6e-bcae-133a974af576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:47.964011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b6595-b65d-4f6e-bcae-133a974af576 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:48.057524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc3e22-92fb-4383-9c6d-cc3b5f2feebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:48.059411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3e22-92fb-4383-9c6d-cc3b5f2feebf service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:48.059916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3e22-92fb-4383-9c6d-cc3b5f2feebf service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:48.060409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3e22-92fb-4383-9c6d-cc3b5f2feebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:48.062520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-99bc3e22-92fb-4383-9c6d-cc3b5f2feebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:48.127826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:48.196802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc3e22-92fb-4383-9c6d-cc3b5f2feebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:48.198671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc3e22-92fb-4383-9c6d-cc3b5f2feebf service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:48.221102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 653/1311] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:03:50.636903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c1b8-b71b-41c6-8dce-8cc15f0cfbaf None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:50.637487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c1b8-b71b-41c6-8dce-8cc15f0cfbaf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:50.637487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c1b8-b71b-41c6-8dce-8cc15f0cfbaf None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:50.638908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 659/1312] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:03:50.643974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cee769-daaa-46b3-b6ba-44dde079f6e2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:50.644423 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cee769-daaa-46b3-b6ba-44dde079f6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:50.644529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cee769-daaa-46b3-b6ba-44dde079f6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:50.727672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-42cee769-daaa-46b3-b6ba-44dde079f6e2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:50.728798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-42cee769-daaa-46b3-b6ba-44dde079f6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:50.832983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 654/1313] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:03:50.872269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248c1b8-b71b-41c6-8dce-8cc15f0cfbaf None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:50.874206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b248c1b8-b71b-41c6-8dce-8cc15f0cfbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:50.995349 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248c1b8-b71b-41c6-8dce-8cc15f0cfbaf None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:51.065035 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:51.067321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9f013-f8ac-42c9-ba4b-6bb63cad5b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:51.067957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9f013-f8ac-42c9-ba4b-6bb63cad5b9b service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:51.068240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9f013-f8ac-42c9-ba4b-6bb63cad5b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:51.068518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9f013-f8ac-42c9-ba4b-6bb63cad5b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:51.070234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d6c9f013-f8ac-42c9-ba4b-6bb63cad5b9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:51.209069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9f013-f8ac-42c9-ba4b-6bb63cad5b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:51.209069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9f013-f8ac-42c9-ba4b-6bb63cad5b9b service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:51.218076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 660/1314] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:53.722081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54ea23-3a89-438b-8e5f-127382f6ca96 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:53.722693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54ea23-3a89-438b-8e5f-127382f6ca96 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:53.722693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54ea23-3a89-438b-8e5f-127382f6ca96 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:53.723918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 655/1315] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:03:53.728880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6243551-354b-4107-9365-bc9785368e56 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:53.729348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6243551-354b-4107-9365-bc9785368e56 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:53.729478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6243551-354b-4107-9365-bc9785368e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:53.800730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d6243551-354b-4107-9365-bc9785368e56 None None] MFA Rules not processed for user `e1f80e64c9e340ae828c1fab2ceed44c`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:53.801982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d6243551-354b-4107-9365-bc9785368e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:53.853839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:53.898720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 661/1316] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:03:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:53.938745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54ea23-3a89-438b-8e5f-127382f6ca96 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:53.940326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8d54ea23-3a89-438b-8e5f-127382f6ca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:53.950612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:54.043436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54ea23-3a89-438b-8e5f-127382f6ca96 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:54.115468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38749529-41b8-4ecb-be50-04f31a93e498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:54.116081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38749529-41b8-4ecb-be50-04f31a93e498 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:54.116363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38749529-41b8-4ecb-be50-04f31a93e498 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:54.116668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38749529-41b8-4ecb-be50-04f31a93e498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:54.118349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-38749529-41b8-4ecb-be50-04f31a93e498 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:54.179811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:54.225641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38749529-41b8-4ecb-be50-04f31a93e498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:54.227074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38749529-41b8-4ecb-be50-04f31a93e498 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:54.245277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 656/1317] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:03:57.119458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9ee4e-b435-4b0e-b07e-94e15e73f884 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:57.120043 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9ee4e-b435-4b0e-b07e-94e15e73f884 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:57.120043 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9ee4e-b435-4b0e-b07e-94e15e73f884 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:57.121953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 662/1318] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:03:57.127188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863d4bc-c91e-4a24-847c-4fb5111d7049 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:57.127608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863d4bc-c91e-4a24-847c-4fb5111d7049 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:57.127749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863d4bc-c91e-4a24-847c-4fb5111d7049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:57.205520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f863d4bc-c91e-4a24-847c-4fb5111d7049 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:03:57.206756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f863d4bc-c91e-4a24-847c-4fb5111d7049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:57.301979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 657/1319] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:03:57.341094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9ee4e-b435-4b0e-b07e-94e15e73f884 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:03:57.342628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f1c9ee4e-b435-4b0e-b07e-94e15e73f884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:57.445097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9ee4e-b435-4b0e-b07e-94e15e73f884 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:03:57.523223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499c819-978c-42a5-88b6-884ff9cf4761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:03:57.523844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499c819-978c-42a5-88b6-884ff9cf4761 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:03:57.524130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499c819-978c-42a5-88b6-884ff9cf4761 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:03:57.524404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499c819-978c-42a5-88b6-884ff9cf4761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:03:57.526068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0499c819-978c-42a5-88b6-884ff9cf4761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:03:57.533950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:03:57.633357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499c819-978c-42a5-88b6-884ff9cf4761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:03:57.634925 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499c819-978c-42a5-88b6-884ff9cf4761 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:03:57.654150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 663/1320] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:03:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:04:00.795375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a015ae-33ea-4606-a82f-260d2ae800b4 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:00.795941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a015ae-33ea-4606-a82f-260d2ae800b4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:00.795941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a015ae-33ea-4606-a82f-260d2ae800b4 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:00.797335 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 658/1321] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:00.802410 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44233686-c42e-44c8-8507-4c6ffe589482 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:00.802849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44233686-c42e-44c8-8507-4c6ffe589482 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:00.802970 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44233686-c42e-44c8-8507-4c6ffe589482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:00.870449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-44233686-c42e-44c8-8507-4c6ffe589482 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:00.871713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-44233686-c42e-44c8-8507-4c6ffe589482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:00.930027 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:00.969451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 664/1322] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:01.006147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a015ae-33ea-4606-a82f-260d2ae800b4 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:01.007695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-65a015ae-33ea-4606-a82f-260d2ae800b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:01.027607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:01.124313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a015ae-33ea-4606-a82f-260d2ae800b4 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:01.203632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1375e1-4b11-4cc5-b4d6-df7afb4ae0cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:01.204266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1375e1-4b11-4cc5-b4d6-df7afb4ae0cb service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:01.204549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1375e1-4b11-4cc5-b4d6-df7afb4ae0cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:01.204824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1375e1-4b11-4cc5-b4d6-df7afb4ae0cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:01.206574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5e1375e1-4b11-4cc5-b4d6-df7afb4ae0cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:01.281007 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:01.319014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1375e1-4b11-4cc5-b4d6-df7afb4ae0cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:01.320460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1375e1-4b11-4cc5-b4d6-df7afb4ae0cb service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:01.338542 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 659/1323] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:03.390841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8aa30-e85a-4651-8b6b-d11fbd008370 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:03.391400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8aa30-e85a-4651-8b6b-d11fbd008370 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:03.391400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8aa30-e85a-4651-8b6b-d11fbd008370 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:03.392816 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 665/1324] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:03.398294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a398be-2bd9-4f14-bedf-5bac5796518e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:03.398718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a398be-2bd9-4f14-bedf-5bac5796518e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:03.398857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a398be-2bd9-4f14-bedf-5bac5796518e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:03.466980 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-27a398be-2bd9-4f14-bedf-5bac5796518e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:03.468229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-27a398be-2bd9-4f14-bedf-5bac5796518e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:03.565019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 660/1325] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:03.601489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8aa30-e85a-4651-8b6b-d11fbd008370 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:03.603112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0eb8aa30-e85a-4651-8b6b-d11fbd008370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:03.709627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8aa30-e85a-4651-8b6b-d11fbd008370 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:03.783173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c9073-a926-4f76-b20d-55bcc9281fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:03.783793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c9073-a926-4f76-b20d-55bcc9281fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:03.784074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c9073-a926-4f76-b20d-55bcc9281fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:03.784345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c9073-a926-4f76-b20d-55bcc9281fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:03.785997 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-674c9073-a926-4f76-b20d-55bcc9281fe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:03.803023 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:03.895910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c9073-a926-4f76-b20d-55bcc9281fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:03.897419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c9073-a926-4f76-b20d-55bcc9281fe9 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:03.915788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 666/1326] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:05.988100 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd5e91-94d5-4160-93d2-91dd2e72c8bb None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:05.988666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd5e91-94d5-4160-93d2-91dd2e72c8bb None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:05.988666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd5e91-94d5-4160-93d2-91dd2e72c8bb None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:05.990211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 661/1327] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:05.995198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d310d-da1a-45cc-9260-89501332f1dc None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:05.996190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d310d-da1a-45cc-9260-89501332f1dc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:05.996290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d310d-da1a-45cc-9260-89501332f1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:06.063919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-348d310d-da1a-45cc-9260-89501332f1dc None None] MFA Rules not processed for user `e1f80e64c9e340ae828c1fab2ceed44c`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:06.065212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-348d310d-da1a-45cc-9260-89501332f1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:06.133364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:06.166177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 667/1328] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:06.204377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd5e91-94d5-4160-93d2-91dd2e72c8bb None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:06.206064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1dfd5e91-94d5-4160-93d2-91dd2e72c8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:06.237522 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:06.316311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd5e91-94d5-4160-93d2-91dd2e72c8bb None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:06.394042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7293451-f055-43a2-bd63-4a6f3f7fac75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:06.394667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7293451-f055-43a2-bd63-4a6f3f7fac75 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:06.394956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7293451-f055-43a2-bd63-4a6f3f7fac75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:06.395224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7293451-f055-43a2-bd63-4a6f3f7fac75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:06.396884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d7293451-f055-43a2-bd63-4a6f3f7fac75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:06.476756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:06.508407 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7293451-f055-43a2-bd63-4a6f3f7fac75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:06.509923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7293451-f055-43a2-bd63-4a6f3f7fac75 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:06.528425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 662/1329] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:09.089365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3989fa-8b68-4022-9e82-8aa0f90bbc6a None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:09.089916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3989fa-8b68-4022-9e82-8aa0f90bbc6a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:09.089916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3989fa-8b68-4022-9e82-8aa0f90bbc6a None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:09.091292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 668/1330] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:09.096410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47e542-a413-4db7-885a-fd81f2456575 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:09.096836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47e542-a413-4db7-885a-fd81f2456575 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:09.096997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47e542-a413-4db7-885a-fd81f2456575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:09.162482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-df47e542-a413-4db7-885a-fd81f2456575 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:09.163694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-df47e542-a413-4db7-885a-fd81f2456575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:09.259218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 663/1331] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:04:09.295819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3989fa-8b68-4022-9e82-8aa0f90bbc6a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:09.297423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ce3989fa-8b68-4022-9e82-8aa0f90bbc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:09.405095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3989fa-8b68-4022-9e82-8aa0f90bbc6a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:09.484254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e000da5-ae06-426e-bfc6-3eaf691db954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:09.484911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e000da5-ae06-426e-bfc6-3eaf691db954 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:09.485348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e000da5-ae06-426e-bfc6-3eaf691db954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:09.485624 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e000da5-ae06-426e-bfc6-3eaf691db954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:09.487560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1e000da5-ae06-426e-bfc6-3eaf691db954 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:09.516051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:09.601617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e000da5-ae06-426e-bfc6-3eaf691db954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:09.603102 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e000da5-ae06-426e-bfc6-3eaf691db954 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:09.622623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 669/1332] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:11.657728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cd31d-0542-4dff-987c-36873a398794 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.658261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cd31d-0542-4dff-987c-36873a398794 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.658261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cd31d-0542-4dff-987c-36873a398794 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.659649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 664/1333] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:11.664653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe436fb-9259-41b5-85e8-f0cb24b88986 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:11.665057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe436fb-9259-41b5-85e8-f0cb24b88986 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:11.665255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe436fb-9259-41b5-85e8-f0cb24b88986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:11.730151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6fe436fb-9259-41b5-85e8-f0cb24b88986 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:11.731180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6fe436fb-9259-41b5-85e8-f0cb24b88986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.803595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:11.825615 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 670/1334] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:11.867730 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cd31d-0542-4dff-987c-36873a398794 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:11.869352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e70cd31d-0542-4dff-987c-36873a398794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:11.912188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:11.974718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cd31d-0542-4dff-987c-36873a398794 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:12.046626 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c13907-dcb9-49c8-aaa2-425dc9a3966a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:12.047240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c13907-dcb9-49c8-aaa2-425dc9a3966a service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:12.047579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c13907-dcb9-49c8-aaa2-425dc9a3966a service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:12.047875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c13907-dcb9-49c8-aaa2-425dc9a3966a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:12.049639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-42c13907-dcb9-49c8-aaa2-425dc9a3966a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:12.133671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:12.158805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c13907-dcb9-49c8-aaa2-425dc9a3966a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:12.160245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c13907-dcb9-49c8-aaa2-425dc9a3966a service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:12.179180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 665/1335] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:04:14.970049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc097071-85c3-41e5-844e-eda77174ea4e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:14.970622 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc097071-85c3-41e5-844e-eda77174ea4e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:14.970622 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc097071-85c3-41e5-844e-eda77174ea4e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:14.971976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 671/1336] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:14.977257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c617c-6596-40a0-bc30-5a9f18034e75 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:14.977652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c617c-6596-40a0-bc30-5a9f18034e75 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:14.977820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c617c-6596-40a0-bc30-5a9f18034e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:15.044306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-bd9c617c-6596-40a0-bc30-5a9f18034e75 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:15.045567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-bd9c617c-6596-40a0-bc30-5a9f18034e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:15.141503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 666/1337] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:15.177304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc097071-85c3-41e5-844e-eda77174ea4e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:15.178870 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dc097071-85c3-41e5-844e-eda77174ea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:15.282626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc097071-85c3-41e5-844e-eda77174ea4e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:15.356110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63f779-e221-47ac-8081-979983e0db5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:15.356730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63f779-e221-47ac-8081-979983e0db5f service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:15.357038 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63f779-e221-47ac-8081-979983e0db5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:15.357317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63f779-e221-47ac-8081-979983e0db5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:15.358959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fc63f779-e221-47ac-8081-979983e0db5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:15.394373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:15.478566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63f779-e221-47ac-8081-979983e0db5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:15.482602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63f779-e221-47ac-8081-979983e0db5f service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:15.504113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 672/1338] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:19.728624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c7110-96be-4177-bb61-0646a801c92c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:19.729242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c7110-96be-4177-bb61-0646a801c92c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:19.729242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c7110-96be-4177-bb61-0646a801c92c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:19.730787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 667/1339] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:19.736352 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7e8e3-2c68-4274-8c33-d2f28772939f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:19.736810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7e8e3-2c68-4274-8c33-d2f28772939f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:19.736902 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7e8e3-2c68-4274-8c33-d2f28772939f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:19.814229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-15f7e8e3-2c68-4274-8c33-d2f28772939f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:19.815529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-15f7e8e3-2c68-4274-8c33-d2f28772939f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:19.911868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:19.924159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 673/1340] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:19.962931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c7110-96be-4177-bb61-0646a801c92c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:19.964624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a42c7110-96be-4177-bb61-0646a801c92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:20.023155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:20.079316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c7110-96be-4177-bb61-0646a801c92c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:20.152420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f542d-a003-4a0b-a6f6-2ae01981e1ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:20.153076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f542d-a003-4a0b-a6f6-2ae01981e1ad service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:20.153364 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f542d-a003-4a0b-a6f6-2ae01981e1ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:20.153637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f542d-a003-4a0b-a6f6-2ae01981e1ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:20.155282 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-398f542d-a003-4a0b-a6f6-2ae01981e1ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:20.248305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:20.264861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f542d-a003-4a0b-a6f6-2ae01981e1ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:20.271264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f542d-a003-4a0b-a6f6-2ae01981e1ad service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:20.287813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 668/1341] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:23.384516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9562-1f45-40cb-a03c-8b749edea363 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:23.385143 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9562-1f45-40cb-a03c-8b749edea363 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:23.385143 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9562-1f45-40cb-a03c-8b749edea363 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:23.387230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 674/1342] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:23.392479 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d21a55-4b60-4714-bc51-f6f05afe3db6 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:23.392945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d21a55-4b60-4714-bc51-f6f05afe3db6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:23.393742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d21a55-4b60-4714-bc51-f6f05afe3db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:23.466093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-b3d21a55-4b60-4714-bc51-f6f05afe3db6 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:23.467301 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b3d21a55-4b60-4714-bc51-f6f05afe3db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:23.566426 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 669/1343] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:23.604922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba9562-1f45-40cb-a03c-8b749edea363 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:23.606495 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6bba9562-1f45-40cb-a03c-8b749edea363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:23.710496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba9562-1f45-40cb-a03c-8b749edea363 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:23.783984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87552560-b0c1-4eee-a35b-6c80ce7f0d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:23.784608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87552560-b0c1-4eee-a35b-6c80ce7f0d8a service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:23.784892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87552560-b0c1-4eee-a35b-6c80ce7f0d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:23.785198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87552560-b0c1-4eee-a35b-6c80ce7f0d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:23.786960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-87552560-b0c1-4eee-a35b-6c80ce7f0d8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:23.831720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:23.906791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87552560-b0c1-4eee-a35b-6c80ce7f0d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:23.908274 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87552560-b0c1-4eee-a35b-6c80ce7f0d8a service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:23.929257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 675/1344] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:28.108905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876b734-d693-4ea9-bfc0-917c4fb3021f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:28.109404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876b734-d693-4ea9-bfc0-917c4fb3021f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:28.109499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876b734-d693-4ea9-bfc0-917c4fb3021f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:28.111138 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 670/1345] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:28.116291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28362185-912a-450e-8e2e-569d22ec53a3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:28.116718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28362185-912a-450e-8e2e-569d22ec53a3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:28.116853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28362185-912a-450e-8e2e-569d22ec53a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:28.191528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-28362185-912a-450e-8e2e-569d22ec53a3 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:28.192758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-28362185-912a-450e-8e2e-569d22ec53a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:28.293039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:28.295608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 676/1346] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:28.349658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ce153-d301-4f8d-8176-083cb941273e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:28.350154 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ce153-d301-4f8d-8176-083cb941273e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:28.350496 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ce153-d301-4f8d-8176-083cb941273e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:28.351949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 671/1347] 10.222.0.86 () {54 vars in 965 bytes} [Thu Jul 28 18:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:28.358850 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f7c7-d63c-41a3-ad5d-0846be6fe642 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:28.359138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f7c7-d63c-41a3-ad5d-0846be6fe642 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:28.369169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f7c7-d63c-41a3-ad5d-0846be6fe642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:28.455136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6711f7c7-d63c-41a3-ad5d-0846be6fe642 None None] MFA Rules not processed for user `b038336582a0435dbd47ad2c09cd23e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:28.456324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6711f7c7-d63c-41a3-ad5d-0846be6fe642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:28.539864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 677/1348] 10.222.0.86 () {58 vars in 1062 bytes} [Thu Jul 28 18:04:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:28.549527 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ce153-d301-4f8d-8176-083cb941273e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:28.551090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f45ce153-d301-4f8d-8176-083cb941273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:28.611220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:28.658529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ce153-d301-4f8d-8176-083cb941273e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:28.733279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c13d1c-6b5e-4975-a028-c56f37fb1dba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:28.733915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c13d1c-6b5e-4975-a028-c56f37fb1dba service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:28.734202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c13d1c-6b5e-4975-a028-c56f37fb1dba service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:28.734481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c13d1c-6b5e-4975-a028-c56f37fb1dba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:28.736417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-61c13d1c-6b5e-4975-a028-c56f37fb1dba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:28.861498 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:28.865152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c13d1c-6b5e-4975-a028-c56f37fb1dba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:28.866597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c13d1c-6b5e-4975-a028-c56f37fb1dba service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:28.885282 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 672/1349] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:34.256678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e85a4-6a02-42d8-bc61-c2164371fa2e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:34.257310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e85a4-6a02-42d8-bc61-c2164371fa2e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:34.257310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e85a4-6a02-42d8-bc61-c2164371fa2e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:34.258945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 678/1350] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:34.265547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33228d17-0239-40e5-90f6-757a17c7c28a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:34.269900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33228d17-0239-40e5-90f6-757a17c7c28a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:34.276377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33228d17-0239-40e5-90f6-757a17c7c28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:34.361163 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-33228d17-0239-40e5-90f6-757a17c7c28a None None] MFA Rules not processed for user `b20564a83953433290aa3fabf884370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:34.362514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-33228d17-0239-40e5-90f6-757a17c7c28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:34.549632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 673/1351] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:04:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:38.538032 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=59 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:38.886319 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509d3ae-2d2c-43e1-a38a-fdb3deb2a7f2 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:38.886647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509d3ae-2d2c-43e1-a38a-fdb3deb2a7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:38.886892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509d3ae-2d2c-43e1-a38a-fdb3deb2a7f2 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:38.888474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 679/1352] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:38.893514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95666198-3f5b-4c8b-ae35-19e9811a5ba2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:38.894540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95666198-3f5b-4c8b-ae35-19e9811a5ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:38.895032 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95666198-3f5b-4c8b-ae35-19e9811a5ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:38.952475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:38.976164 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-95666198-3f5b-4c8b-ae35-19e9811a5ba2 None None] MFA Rules not processed for user `b20564a83953433290aa3fabf884370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:38.977430 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-95666198-3f5b-4c8b-ae35-19e9811a5ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:39.103917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 674/1353] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:40.326039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b1eeb-4587-4be5-b63d-bfa823a271aa None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:40.326590 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b1eeb-4587-4be5-b63d-bfa823a271aa None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:40.326590 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b1eeb-4587-4be5-b63d-bfa823a271aa None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:40.327972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 680/1354] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:40.333041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8fe98-89e3-4e76-9283-d56f1c3b60ce None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:40.333452 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8fe98-89e3-4e76-9283-d56f1c3b60ce None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:40.333610 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8fe98-89e3-4e76-9283-d56f1c3b60ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:40.410901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1fc8fe98-89e3-4e76-9283-d56f1c3b60ce None None] MFA Rules not processed for user `b20564a83953433290aa3fabf884370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:40.412114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1fc8fe98-89e3-4e76-9283-d56f1c3b60ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:40.465297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:40.535667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 675/1355] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:04:43.957291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b92800-c265-4a03-b6fc-a93f1448adfc None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:43.957840 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b92800-c265-4a03-b6fc-a93f1448adfc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:43.957958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b92800-c265-4a03-b6fc-a93f1448adfc None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:43.959444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 681/1356] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:43.966667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be9b10-9e86-4aab-ba68-99a210d87cd4 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:43.970298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be9b10-9e86-4aab-ba68-99a210d87cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:43.970797 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be9b10-9e86-4aab-ba68-99a210d87cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:44.067591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-81be9b10-9e86-4aab-ba68-99a210d87cd4 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:44.070052 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-81be9b10-9e86-4aab-ba68-99a210d87cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:44.181839 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 676/1357] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:44.238388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00230a-fd97-4e81-b4fa-92ca79268ff8 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:44.238388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00230a-fd97-4e81-b4fa-92ca79268ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:44.238388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00230a-fd97-4e81-b4fa-92ca79268ff8 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:44.240113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 682/1358] 10.222.0.86 () {54 vars in 962 bytes} [Thu Jul 28 18:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:44.247250 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a3f8-5585-4193-8011-8bd031b870aa None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:44.247588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a3f8-5585-4193-8011-8bd031b870aa None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:44.247822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a3f8-5585-4193-8011-8bd031b870aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:44.267277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:44.333086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e205a3f8-5585-4193-8011-8bd031b870aa None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:44.334394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e205a3f8-5585-4193-8011-8bd031b870aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:44.468911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 677/1359] 10.222.0.86 () {58 vars in 1059 bytes} [Thu Jul 28 18:04:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:44.476690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00230a-fd97-4e81-b4fa-92ca79268ff8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:44.478437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5f00230a-fd97-4e81-b4fa-92ca79268ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:44.652949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00230a-fd97-4e81-b4fa-92ca79268ff8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:44.735252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:44.777409 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332183fd-f82d-405d-9fbc-f100a150fb7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:44.778039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332183fd-f82d-405d-9fbc-f100a150fb7c service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:44.778325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332183fd-f82d-405d-9fbc-f100a150fb7c service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:44.778599 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332183fd-f82d-405d-9fbc-f100a150fb7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:44.780227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-332183fd-f82d-405d-9fbc-f100a150fb7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:44.878300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332183fd-f82d-405d-9fbc-f100a150fb7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:44.879937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332183fd-f82d-405d-9fbc-f100a150fb7c service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:44.909029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 683/1360] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:47.148736 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6de9a2-f45a-4641-a809-69294af06c5d None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:47.149367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6de9a2-f45a-4641-a809-69294af06c5d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:47.149367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6de9a2-f45a-4641-a809-69294af06c5d None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:47.150772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 678/1361] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:47.155765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802136e5-be88-4314-82ac-944ff5ddd67d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:47.156168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802136e5-be88-4314-82ac-944ff5ddd67d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:47.156314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802136e5-be88-4314-82ac-944ff5ddd67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:47.226895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-802136e5-be88-4314-82ac-944ff5ddd67d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:47.228100 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-802136e5-be88-4314-82ac-944ff5ddd67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:47.267969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:47.337528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 684/1362] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:47.388728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6de9a2-f45a-4641-a809-69294af06c5d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:47.390545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fa6de9a2-f45a-4641-a809-69294af06c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:47.471673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:47.567820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6de9a2-f45a-4641-a809-69294af06c5d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:47.702865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497fbde-3968-4fae-a2db-aa0137640089 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:47.704050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497fbde-3968-4fae-a2db-aa0137640089 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:47.705351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497fbde-3968-4fae-a2db-aa0137640089 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:47.705351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497fbde-3968-4fae-a2db-aa0137640089 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:47.707868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f497fbde-3968-4fae-a2db-aa0137640089 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:47.758491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:47.835213 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497fbde-3968-4fae-a2db-aa0137640089 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:47.836941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497fbde-3968-4fae-a2db-aa0137640089 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:47.858655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 679/1363] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:50.339169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58708a-3ec7-48c9-8dde-6e56f6db8633 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:50.339703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58708a-3ec7-48c9-8dde-6e56f6db8633 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:50.339703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58708a-3ec7-48c9-8dde-6e56f6db8633 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:50.341388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 685/1364] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:50.346806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7284f0f-aa7e-42e1-a666-17c14ab7a5ac None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:50.347321 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7284f0f-aa7e-42e1-a666-17c14ab7a5ac None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:50.347418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7284f0f-aa7e-42e1-a666-17c14ab7a5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:50.441251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-b7284f0f-aa7e-42e1-a666-17c14ab7a5ac None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:50.442481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b7284f0f-aa7e-42e1-a666-17c14ab7a5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:50.544661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:50.547520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 680/1365] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:50.594650 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02b2d6-7ca5-495d-8640-cc2b92f5f42e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:50.594874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02b2d6-7ca5-495d-8640-cc2b92f5f42e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:50.595335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02b2d6-7ca5-495d-8640-cc2b92f5f42e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:50.596698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 686/1366] 10.222.0.86 () {54 vars in 962 bytes} [Thu Jul 28 18:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:50.602446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d97f76-e7fb-42db-9df2-fc35de98a017 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:50.605075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d97f76-e7fb-42db-9df2-fc35de98a017 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:50.605075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d97f76-e7fb-42db-9df2-fc35de98a017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:50.683333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d4d97f76-e7fb-42db-9df2-fc35de98a017 None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:50.684550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d4d97f76-e7fb-42db-9df2-fc35de98a017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:50.810781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 681/1367] 10.222.0.86 () {58 vars in 1059 bytes} [Thu Jul 28 18:04:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:50.818639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02b2d6-7ca5-495d-8640-cc2b92f5f42e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:50.820181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fd02b2d6-7ca5-495d-8640-cc2b92f5f42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:50.978589 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02b2d6-7ca5-495d-8640-cc2b92f5f42e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:51.000468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:51.097512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75794ea5-01f1-45d1-9efb-569af13a843f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:51.098138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75794ea5-01f1-45d1-9efb-569af13a843f service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:51.098419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75794ea5-01f1-45d1-9efb-569af13a843f service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:51.098689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75794ea5-01f1-45d1-9efb-569af13a843f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:51.100663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-75794ea5-01f1-45d1-9efb-569af13a843f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:51.197989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75794ea5-01f1-45d1-9efb-569af13a843f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:51.199601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75794ea5-01f1-45d1-9efb-569af13a843f service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:51.219088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 687/1368] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:52.248795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc488f56-fea9-4176-89c0-018001a4ca65 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:52.249403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc488f56-fea9-4176-89c0-018001a4ca65 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:52.249403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc488f56-fea9-4176-89c0-018001a4ca65 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:52.251003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 682/1369] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:52.256231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827101f-b39e-4a77-ae65-1e5198252823 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:52.257086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827101f-b39e-4a77-ae65-1e5198252823 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:52.257595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827101f-b39e-4a77-ae65-1e5198252823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:52.290025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:52.325910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8827101f-b39e-4a77-ae65-1e5198252823 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:52.327266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8827101f-b39e-4a77-ae65-1e5198252823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:52.437317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 688/1370] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:52.786337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358864-d65d-4948-9144-f46bd70120d7 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:52.786755 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358864-d65d-4948-9144-f46bd70120d7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:52.787182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358864-d65d-4948-9144-f46bd70120d7 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:52.788719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 683/1371] 10.222.0.86 () {54 vars in 964 bytes} [Thu Jul 28 18:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:52.794149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bdc7f-d0e1-45d9-8f1e-062c79d50170 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:52.794545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bdc7f-d0e1-45d9-8f1e-062c79d50170 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:52.794930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bdc7f-d0e1-45d9-8f1e-062c79d50170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:52.879620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-bc8bdc7f-d0e1-45d9-8f1e-062c79d50170 None None] MFA Rules not processed for user `9dbaf1e343474322b2567e2dfda99d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:52.881106 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bc8bdc7f-d0e1-45d9-8f1e-062c79d50170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:52.930318 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:52.960984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 689/1372] 10.222.0.86 () {58 vars in 1061 bytes} [Thu Jul 28 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:52.968979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81358864-d65d-4948-9144-f46bd70120d7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:52.970635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-81358864-d65d-4948-9144-f46bd70120d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:53.027296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:53.084863 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81358864-d65d-4948-9144-f46bd70120d7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:53.170123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc20d64-ae2c-489e-900d-a0e9e4aa06c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:53.170755 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc20d64-ae2c-489e-900d-a0e9e4aa06c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:53.171041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc20d64-ae2c-489e-900d-a0e9e4aa06c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:53.171321 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc20d64-ae2c-489e-900d-a0e9e4aa06c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:53.173017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4cc20d64-ae2c-489e-900d-a0e9e4aa06c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:53.280771 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:53.290922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc20d64-ae2c-489e-900d-a0e9e4aa06c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:53.292417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc20d64-ae2c-489e-900d-a0e9e4aa06c1 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:53.311649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 684/1373] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:53.489272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec8501-2498-4b50-a693-7d9891f39047 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:53.489785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec8501-2498-4b50-a693-7d9891f39047 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:53.489785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec8501-2498-4b50-a693-7d9891f39047 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:53.491867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 690/1374] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:53.496296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0bc79-1b12-4b58-b021-463f055d0ac2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:53.496591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0bc79-1b12-4b58-b021-463f055d0ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:53.496877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0bc79-1b12-4b58-b021-463f055d0ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:53.567605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-93e0bc79-1b12-4b58-b021-463f055d0ac2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:53.568817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-93e0bc79-1b12-4b58-b021-463f055d0ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:53.669047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 685/1375] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:53.717112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec8501-2498-4b50-a693-7d9891f39047 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:53.717112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-afec8501-2498-4b50-a693-7d9891f39047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:53.875647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec8501-2498-4b50-a693-7d9891f39047 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:53.928093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:53.992214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb595dc9-3e69-4db5-baa9-604f4487e2f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:53.992839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb595dc9-3e69-4db5-baa9-604f4487e2f7 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:53.993147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb595dc9-3e69-4db5-baa9-604f4487e2f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:53.993426 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb595dc9-3e69-4db5-baa9-604f4487e2f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:53.995091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-eb595dc9-3e69-4db5-baa9-604f4487e2f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:54.086087 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb595dc9-3e69-4db5-baa9-604f4487e2f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:54.087714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb595dc9-3e69-4db5-baa9-604f4487e2f7 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:54.106408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 691/1376] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:56.268120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac96789-986b-4ca9-b95f-55ce1cb6c245 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:56.268583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac96789-986b-4ca9-b95f-55ce1cb6c245 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:56.268700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac96789-986b-4ca9-b95f-55ce1cb6c245 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:56.270091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 686/1377] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:56.275447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33d311-88fa-49d9-9983-6137ec54bafb None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:56.275910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33d311-88fa-49d9-9983-6137ec54bafb None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:56.276094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33d311-88fa-49d9-9983-6137ec54bafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:56.348297 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1c33d311-88fa-49d9-9983-6137ec54bafb None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:56.349701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1c33d311-88fa-49d9-9983-6137ec54bafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:56.356879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:56.458960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 692/1378] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:56.508376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac96789-986b-4ca9-b95f-55ce1cb6c245 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:56.510062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cac96789-986b-4ca9-b95f-55ce1cb6c245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:56.582061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:56.668429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac96789-986b-4ca9-b95f-55ce1cb6c245 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:56.784708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038eb8ed-9ed9-4574-bec3-021ccbffdfb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:56.785366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eb8ed-9ed9-4574-bec3-021ccbffdfb3 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:56.785654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eb8ed-9ed9-4574-bec3-021ccbffdfb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:56.785933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eb8ed-9ed9-4574-bec3-021ccbffdfb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:56.787591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-038eb8ed-9ed9-4574-bec3-021ccbffdfb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:56.806474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:56.890371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038eb8ed-9ed9-4574-bec3-021ccbffdfb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:56.891995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038eb8ed-9ed9-4574-bec3-021ccbffdfb3 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:56.911526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 687/1379] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:04:59.131997 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef947d47-d89d-49eb-96b6-1b44819515e9 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:59.132558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef947d47-d89d-49eb-96b6-1b44819515e9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:59.132558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef947d47-d89d-49eb-96b6-1b44819515e9 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:59.138935 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 693/1380] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:04:59.140292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474bc15-aab4-48a3-b82b-b2b764aaaf46 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:59.140770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474bc15-aab4-48a3-b82b-b2b764aaaf46 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:59.140951 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474bc15-aab4-48a3-b82b-b2b764aaaf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:59.213315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d474bc15-aab4-48a3-b82b-b2b764aaaf46 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:04:59.214534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d474bc15-aab4-48a3-b82b-b2b764aaaf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:59.315963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:59.324776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 688/1381] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:04:59.367027 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef947d47-d89d-49eb-96b6-1b44819515e9 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:04:59.368641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ef947d47-d89d-49eb-96b6-1b44819515e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:59.535595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:59.548079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef947d47-d89d-49eb-96b6-1b44819515e9 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:04:59.670503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f124283-3b83-4c36-948e-d1e4246df6ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:04:59.671085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f124283-3b83-4c36-948e-d1e4246df6ae service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:04:59.671366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f124283-3b83-4c36-948e-d1e4246df6ae service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:04:59.671640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f124283-3b83-4c36-948e-d1e4246df6ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:04:59.673305 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1f124283-3b83-4c36-948e-d1e4246df6ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:04:59.774273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f124283-3b83-4c36-948e-d1e4246df6ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:04:59.775740 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f124283-3b83-4c36-948e-d1e4246df6ae service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:04:59.789330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:04:59.799215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 694/1382] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:04:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:02.012866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad1924-bc99-4202-8fe8-e2e2feb38708 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:02.013623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad1924-bc99-4202-8fe8-e2e2feb38708 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:02.013623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad1924-bc99-4202-8fe8-e2e2feb38708 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:02.014822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 689/1383] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:02.020654 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb970864-f8ba-40bc-aaf1-8cdf9e8e87ca None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:02.021175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb970864-f8ba-40bc-aaf1-8cdf9e8e87ca None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:02.021544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb970864-f8ba-40bc-aaf1-8cdf9e8e87ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:02.109229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-fb970864-f8ba-40bc-aaf1-8cdf9e8e87ca None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:02.110709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fb970864-f8ba-40bc-aaf1-8cdf9e8e87ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:02.210058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 695/1384] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:02.255181 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad1924-bc99-4202-8fe8-e2e2feb38708 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:02.257504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0bad1924-bc99-4202-8fe8-e2e2feb38708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:02.409392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad1924-bc99-4202-8fe8-e2e2feb38708 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:02.482987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:02.534411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f76c3b-1d48-4fdd-be5c-fda85b2396fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:02.535036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f76c3b-1d48-4fdd-be5c-fda85b2396fd service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:02.535320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f76c3b-1d48-4fdd-be5c-fda85b2396fd service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:02.535597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f76c3b-1d48-4fdd-be5c-fda85b2396fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:02.537286 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c2f76c3b-1d48-4fdd-be5c-fda85b2396fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:02.628450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f76c3b-1d48-4fdd-be5c-fda85b2396fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:02.630177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f76c3b-1d48-4fdd-be5c-fda85b2396fd service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:02.657212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 690/1385] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:05.561622 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185836-7eb5-4e3e-bab0-13aa3ec5ed3b None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:05.562168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185836-7eb5-4e3e-bab0-13aa3ec5ed3b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:05.562447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185836-7eb5-4e3e-bab0-13aa3ec5ed3b None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:05.564596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 696/1386] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:05:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:05.570217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dbca0-adee-4fee-aa2e-ec585c00d8b0 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:05.570679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dbca0-adee-4fee-aa2e-ec585c00d8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:05.570826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dbca0-adee-4fee-aa2e-ec585c00d8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:05.645375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-571dbca0-adee-4fee-aa2e-ec585c00d8b0 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:05.647119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-571dbca0-adee-4fee-aa2e-ec585c00d8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:05.699197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:05.777397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 691/1387] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:05.826783 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f185836-7eb5-4e3e-bab0-13aa3ec5ed3b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:05.828618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1f185836-7eb5-4e3e-bab0-13aa3ec5ed3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:05.923884 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:06.001057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f185836-7eb5-4e3e-bab0-13aa3ec5ed3b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:06.133348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3391725-0267-4052-94e1-33dff14c6513 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:06.134004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3391725-0267-4052-94e1-33dff14c6513 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:06.134285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3391725-0267-4052-94e1-33dff14c6513 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:06.134568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3391725-0267-4052-94e1-33dff14c6513 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:06.136303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e3391725-0267-4052-94e1-33dff14c6513 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:06.181772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:06.235468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3391725-0267-4052-94e1-33dff14c6513 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:06.237283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3391725-0267-4052-94e1-33dff14c6513 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:06.257473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 697/1388] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:08.476379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161ab69-e19d-4471-b74d-b14074f9ec98 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:08.477553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161ab69-e19d-4471-b74d-b14074f9ec98 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:08.477553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161ab69-e19d-4471-b74d-b14074f9ec98 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:08.479277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 692/1389] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:08.486101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef95e0-1d68-425b-ba86-3da7ef99b23d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:08.486559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef95e0-1d68-425b-ba86-3da7ef99b23d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:08.486798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef95e0-1d68-425b-ba86-3da7ef99b23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:08.561956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a7ef95e0-1d68-425b-ba86-3da7ef99b23d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:08.563154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a7ef95e0-1d68-425b-ba86-3da7ef99b23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:08.659576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 698/1390] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:08.705365 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161ab69-e19d-4471-b74d-b14074f9ec98 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:08.707094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c161ab69-e19d-4471-b74d-b14074f9ec98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:08.771764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aefb6a-3789-4ba6-b342-2f7ff79b969c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:08.772429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aefb6a-3789-4ba6-b342-2f7ff79b969c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:08.772807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aefb6a-3789-4ba6-b342-2f7ff79b969c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:08.774612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 699/1391] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 18:05:08.779567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6f09-8a3a-48fb-a775-80aeffa73ac4 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:08.779860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6f09-8a3a-48fb-a775-80aeffa73ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:08.780141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6f09-8a3a-48fb-a775-80aeffa73ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:08.796039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:08.848368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d1ec6f09-8a3a-48fb-a775-80aeffa73ac4 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:08.849630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d1ec6f09-8a3a-48fb-a775-80aeffa73ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:08.872633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161ab69-e19d-4471-b74d-b14074f9ec98 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:08.890287 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:08.952343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 700/1392] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:05:08.991888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b529bd2-d434-417f-8aea-c43dd4df33ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:08.992523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b529bd2-d434-417f-8aea-c43dd4df33ba service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:08.992807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b529bd2-d434-417f-8aea-c43dd4df33ba service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:08.993226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b529bd2-d434-417f-8aea-c43dd4df33ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:08.994928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6b529bd2-d434-417f-8aea-c43dd4df33ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:09.014721 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb3146-7d1f-4dc0-81ac-1de073d92ca1 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:09.015022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb3146-7d1f-4dc0-81ac-1de073d92ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:09.015297 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb3146-7d1f-4dc0-81ac-1de073d92ca1 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:09.016672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 701/1393] 10.222.0.86 () {54 vars in 966 bytes} [Thu Jul 28 18:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:09.021924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd36862-8bc7-4ff3-88be-262e39854ea5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:09.022207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd36862-8bc7-4ff3-88be-262e39854ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:09.022482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd36862-8bc7-4ff3-88be-262e39854ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:09.090756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b529bd2-d434-417f-8aea-c43dd4df33ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:09.092385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b529bd2-d434-417f-8aea-c43dd4df33ba service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:09.099795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7bd36862-8bc7-4ff3-88be-262e39854ea5 None None] MFA Rules not processed for user `c47f1552e73648ceb8a421c0543c2974`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:09.101138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7bd36862-8bc7-4ff3-88be-262e39854ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:09.110987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 693/1394] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:09.137581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:09.178513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 702/1395] 10.222.0.86 () {58 vars in 1063 bytes} [Thu Jul 28 18:05:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:05:09.186108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b529bd2-d434-417f-8aea-c43dd4df33ba service nova] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:09.187304 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6b529bd2-d434-417f-8aea-c43dd4df33ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:09.216599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:09.266270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b529bd2-d434-417f-8aea-c43dd4df33ba service nova] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:09.349119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b9b05-112c-4c33-9c8f-8b08150cf304 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47f1552e73648ceb8a421c0543c2974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:09.349765 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b9b05-112c-4c33-9c8f-8b08150cf304 service glance] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:09.350047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b9b05-112c-4c33-9c8f-8b08150cf304 service glance] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:09.350324 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b9b05-112c-4c33-9c8f-8b08150cf304 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:09.352033 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5c5b9b05-112c-4c33-9c8f-8b08150cf304 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:09.463888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:09.479201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b9b05-112c-4c33-9c8f-8b08150cf304 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:09.479201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b9b05-112c-4c33-9c8f-8b08150cf304 service glance] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:09.496883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 694/1396] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:10.422830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4928-a91f-4352-a16f-39a2dac3cf9e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:10.423304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4928-a91f-4352-a16f-39a2dac3cf9e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:10.423385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4928-a91f-4352-a16f-39a2dac3cf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:10.503004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-abca4928-a91f-4352-a16f-39a2dac3cf9e None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:10.504209 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-abca4928-a91f-4352-a16f-39a2dac3cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:10.597097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 703/1397] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:05:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:10.639947 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07708e-d87c-43f3-b174-cea38f2ee041 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:10.640435 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07708e-d87c-43f3-b174-cea38f2ee041 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:10.640528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07708e-d87c-43f3-b174-cea38f2ee041 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:10.641921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 695/1398] 10.222.0.86 () {54 vars in 943 bytes} [Thu Jul 28 18:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:10.648140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c4b8a-f511-4434-99f4-af9fef49d8b8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:10.648431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c4b8a-f511-4434-99f4-af9fef49d8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:10.648709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c4b8a-f511-4434-99f4-af9fef49d8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:10.721927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:10.747964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-845c4b8a-f511-4434-99f4-af9fef49d8b8 None None] MFA Rules not processed for user `3077d844257c48b3b4ac2db9177fb6df`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:10.749502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-845c4b8a-f511-4434-99f4-af9fef49d8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:10.866123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 704/1399] 10.222.0.86 () {58 vars in 1040 bytes} [Thu Jul 28 18:05:10 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:10.875922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07708e-d87c-43f3-b174-cea38f2ee041 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:10.877541 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0d07708e-d87c-43f3-b174-cea38f2ee041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:11.021305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07708e-d87c-43f3-b174-cea38f2ee041 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:11.093047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:11.133010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d218ec-bc68-43fd-8871-9278385a7c72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:11.134274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d218ec-bc68-43fd-8871-9278385a7c72 service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:11.134757 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d218ec-bc68-43fd-8871-9278385a7c72 service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:11.135212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d218ec-bc68-43fd-8871-9278385a7c72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:11.137468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e9d218ec-bc68-43fd-8871-9278385a7c72 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:11.222192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d218ec-bc68-43fd-8871-9278385a7c72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:11.223924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d218ec-bc68-43fd-8871-9278385a7c72 service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:11.232529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 696/1400] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:05:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:05:11.312446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46063375-dd9d-4b8b-a2b8-57364faf023c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:11.312758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46063375-dd9d-4b8b-a2b8-57364faf023c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:11.313069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46063375-dd9d-4b8b-a2b8-57364faf023c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:11.314453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 705/1401] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:11.319866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b6416-8604-4ae0-b336-ce94b7e1174e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:11.320244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b6416-8604-4ae0-b336-ce94b7e1174e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:11.337120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b6416-8604-4ae0-b336-ce94b7e1174e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:11.388158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d12b6416-8604-4ae0-b336-ce94b7e1174e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:11.389343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d12b6416-8604-4ae0-b336-ce94b7e1174e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:11.442665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:11.487911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 697/1402] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:11.531703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46063375-dd9d-4b8b-a2b8-57364faf023c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:11.533437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-46063375-dd9d-4b8b-a2b8-57364faf023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:11.640988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:11.683343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46063375-dd9d-4b8b-a2b8-57364faf023c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:11.809876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864a51f-4c98-425f-bc8c-44b93814d6c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:11.810507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864a51f-4c98-425f-bc8c-44b93814d6c0 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:11.810795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864a51f-4c98-425f-bc8c-44b93814d6c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:11.811074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864a51f-4c98-425f-bc8c-44b93814d6c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:11.812729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a864a51f-4c98-425f-bc8c-44b93814d6c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:11.887771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:11.907833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864a51f-4c98-425f-bc8c-44b93814d6c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:11.909549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864a51f-4c98-425f-bc8c-44b93814d6c0 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:11.931645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 706/1403] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:14.194530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08146ab5-3e38-446a-8beb-0df2013a49ad None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:14.195104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08146ab5-3e38-446a-8beb-0df2013a49ad None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:14.195104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08146ab5-3e38-446a-8beb-0df2013a49ad None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:14.197042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 698/1404] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:14.202320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09fbcc-3beb-47cb-bf15-cf2f8df3d95b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:14.202800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09fbcc-3beb-47cb-bf15-cf2f8df3d95b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:14.202885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09fbcc-3beb-47cb-bf15-cf2f8df3d95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:14.291606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-df09fbcc-3beb-47cb-bf15-cf2f8df3d95b None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:14.292855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-df09fbcc-3beb-47cb-bf15-cf2f8df3d95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:14.404702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 707/1405] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:14.462662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08146ab5-3e38-446a-8beb-0df2013a49ad None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:14.464207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-08146ab5-3e38-446a-8beb-0df2013a49ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:14.657158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08146ab5-3e38-446a-8beb-0df2013a49ad None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:14.705930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:14.810601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a10e6-af03-4430-ab1d-91eaf0ee7dad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:14.811812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a10e6-af03-4430-ab1d-91eaf0ee7dad service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:14.812269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a10e6-af03-4430-ab1d-91eaf0ee7dad service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:14.812815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a10e6-af03-4430-ab1d-91eaf0ee7dad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:14.815442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0e3a10e6-af03-4430-ab1d-91eaf0ee7dad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:14.938837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a10e6-af03-4430-ab1d-91eaf0ee7dad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:14.940454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a10e6-af03-4430-ab1d-91eaf0ee7dad service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:14.962993 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 699/1406] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:17.259159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c0679-559c-43c4-9f50-24b368f28d3a None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:17.259712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c0679-559c-43c4-9f50-24b368f28d3a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:17.259812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c0679-559c-43c4-9f50-24b368f28d3a None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:17.261683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 708/1407] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:17.266965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a7437-d9e6-48b8-8151-a8de64b85102 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:17.267458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a7437-d9e6-48b8-8151-a8de64b85102 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:17.267458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a7437-d9e6-48b8-8151-a8de64b85102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:17.333937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:17.349488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-880a7437-d9e6-48b8-8151-a8de64b85102 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:17.351469 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-880a7437-d9e6-48b8-8151-a8de64b85102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:17.477046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 700/1408] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:17.518222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c0679-559c-43c4-9f50-24b368f28d3a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:17.519764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9e0c0679-559c-43c4-9f50-24b368f28d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:17.575630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:17.688313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c0679-559c-43c4-9f50-24b368f28d3a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:17.827355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04654e-f5e6-4204-a929-e3761b2b49d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:17.832419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04654e-f5e6-4204-a929-e3761b2b49d8 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:17.832419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04654e-f5e6-4204-a929-e3761b2b49d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:17.832419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04654e-f5e6-4204-a929-e3761b2b49d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:17.832419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8f04654e-f5e6-4204-a929-e3761b2b49d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:17.841017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:17.939066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04654e-f5e6-4204-a929-e3761b2b49d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:17.940735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04654e-f5e6-4204-a929-e3761b2b49d8 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:17.967681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 709/1409] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:27.469568 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=46 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:27.961842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=41 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:35.615433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e09916-3ffc-4efc-a54c-e8b2b5f5e166 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:35.615995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e09916-3ffc-4efc-a54c-e8b2b5f5e166 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:35.615995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e09916-3ffc-4efc-a54c-e8b2b5f5e166 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:35.617442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 701/1410] 10.222.0.86 () {54 vars in 918 bytes} [Thu Jul 28 18:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:35.622506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d920-1b2c-4d30-b65a-78fa65078580 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:35.622937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d920-1b2c-4d30-b65a-78fa65078580 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:35.623070 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d920-1b2c-4d30-b65a-78fa65078580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:35.701799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b4e2d920-1b2c-4d30-b65a-78fa65078580 None None] MFA Rules not processed for user `b20564a83953433290aa3fabf884370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:35.703731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b4e2d920-1b2c-4d30-b65a-78fa65078580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:35.809783 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 710/1411] 10.222.0.86 () {58 vars in 1015 bytes} [Thu Jul 28 18:05:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:35.824597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf167cf3-ca81-422a-9c38-fb4fd20dbf16 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:35.824879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf167cf3-ca81-422a-9c38-fb4fd20dbf16 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:35.825185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf167cf3-ca81-422a-9c38-fb4fd20dbf16 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:35.826550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 702/1412] 10.222.0.86 () {54 vars in 918 bytes} [Thu Jul 28 18:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:35.831291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a5a20-71aa-4c6e-97ad-2a6accb3af77 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:35.831589 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a5a20-71aa-4c6e-97ad-2a6accb3af77 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:35.831872 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a5a20-71aa-4c6e-97ad-2a6accb3af77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:35.906559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:35.909712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-aa7a5a20-71aa-4c6e-97ad-2a6accb3af77 None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:35.910923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-aa7a5a20-71aa-4c6e-97ad-2a6accb3af77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:36.030797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 711/1413] 10.222.0.86 () {58 vars in 1015 bytes} [Thu Jul 28 18:05:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:38.493991 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a4f6b-3afd-4422-8f19-6ff40366a777 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:38.494536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a4f6b-3afd-4422-8f19-6ff40366a777 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:38.494646 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a4f6b-3afd-4422-8f19-6ff40366a777 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:38.495951 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:38.501235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5fcd1-0068-4d5a-8d77-9bf255835f52 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:38.501642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5fcd1-0068-4d5a-8d77-9bf255835f52 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:38.501799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5fcd1-0068-4d5a-8d77-9bf255835f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:38.568402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-dcd5fcd1-0068-4d5a-8d77-9bf255835f52 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:38.569637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dcd5fcd1-0068-4d5a-8d77-9bf255835f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:38.624356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:38.668285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:38.712537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a4f6b-3afd-4422-8f19-6ff40366a777 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:38.714086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9c1a4f6b-3afd-4422-8f19-6ff40366a777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:38.858729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a4f6b-3afd-4422-8f19-6ff40366a777 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:38.930297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:38.971558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d2700-3c04-4b07-8112-4e7be4d18530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:38.972110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d2700-3c04-4b07-8112-4e7be4d18530 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:38.972403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d2700-3c04-4b07-8112-4e7be4d18530 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:38.972666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d2700-3c04-4b07-8112-4e7be4d18530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:38.974326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b31d2700-3c04-4b07-8112-4e7be4d18530 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:39.062309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d2700-3c04-4b07-8112-4e7be4d18530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:39.063915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d2700-3c04-4b07-8112-4e7be4d18530 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:39.082222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 704/1416] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:40.011839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd700a02-d1c6-458a-9c5a-78b4b01975d9 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:40.012495 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd700a02-d1c6-458a-9c5a-78b4b01975d9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:40.012495 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd700a02-d1c6-458a-9c5a-78b4b01975d9 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:40.013811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:40.018788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600c42f-23f1-4d35-9dcf-6bf9df3b22f2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:40.019193 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600c42f-23f1-4d35-9dcf-6bf9df3b22f2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:40.019349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600c42f-23f1-4d35-9dcf-6bf9df3b22f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:40.094482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0600c42f-23f1-4d35-9dcf-6bf9df3b22f2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:40.095716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0600c42f-23f1-4d35-9dcf-6bf9df3b22f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:40.122556 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:40.194274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:40.349717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd700a02-d1c6-458a-9c5a-78b4b01975d9 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:40.351785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dd700a02-d1c6-458a-9c5a-78b4b01975d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:40.498008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd700a02-d1c6-458a-9c5a-78b4b01975d9 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:40.537478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:40.611045 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcc02e-d30d-4142-a6cd-489d7cd56e7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:40.611593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcc02e-d30d-4142-a6cd-489d7cd56e7f service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:40.611873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcc02e-d30d-4142-a6cd-489d7cd56e7f service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:40.612156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcc02e-d30d-4142-a6cd-489d7cd56e7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:40.613834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6adcc02e-d30d-4142-a6cd-489d7cd56e7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:40.703127 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcc02e-d30d-4142-a6cd-489d7cd56e7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:40.704715 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcc02e-d30d-4142-a6cd-489d7cd56e7f service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:40.722995 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 714/1419] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:41.609243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb1869-52c9-410a-8249-ef73ca37cabb None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:41.609812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb1869-52c9-410a-8249-ef73ca37cabb None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:41.609812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb1869-52c9-410a-8249-ef73ca37cabb None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:41.611177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:41.616232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfd89b-976d-4a73-84e6-f611720ed37d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:41.616679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfd89b-976d-4a73-84e6-f611720ed37d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:41.616912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfd89b-976d-4a73-84e6-f611720ed37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:41.676474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:41.703571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1abfd89b-976d-4a73-84e6-f611720ed37d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:41.704790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1abfd89b-976d-4a73-84e6-f611720ed37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:41.799711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:41.962327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb1869-52c9-410a-8249-ef73ca37cabb None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:41.963796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c2cb1869-52c9-410a-8249-ef73ca37cabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:42.126955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb1869-52c9-410a-8249-ef73ca37cabb None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:42.139613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:42.239841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c75f05-3ac1-4834-a204-ced304b8d9b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:42.240441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75f05-3ac1-4834-a204-ced304b8d9b4 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:42.240719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75f05-3ac1-4834-a204-ced304b8d9b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:42.241031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75f05-3ac1-4834-a204-ced304b8d9b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:42.243156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d0c75f05-3ac1-4834-a204-ced304b8d9b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:42.331694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c75f05-3ac1-4834-a204-ced304b8d9b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:42.333329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c75f05-3ac1-4834-a204-ced304b8d9b4 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:42.351014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 707/1422] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:43.216154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642a2d0-9cf2-4fbb-a6fb-b52843fac22a None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:43.216795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642a2d0-9cf2-4fbb-a6fb-b52843fac22a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:43.216795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642a2d0-9cf2-4fbb-a6fb-b52843fac22a None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:43.218156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:43.223311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67647f0d-6e28-496c-93a1-debb32aed31c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:43.223782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67647f0d-6e28-496c-93a1-debb32aed31c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:43.223895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67647f0d-6e28-496c-93a1-debb32aed31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:43.244499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:43.305322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-67647f0d-6e28-496c-93a1-debb32aed31c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:43.306568 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-67647f0d-6e28-496c-93a1-debb32aed31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:43.403374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:05:43.558265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642a2d0-9cf2-4fbb-a6fb-b52843fac22a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:43.559789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3642a2d0-9cf2-4fbb-a6fb-b52843fac22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:43.669449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:43.705583 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642a2d0-9cf2-4fbb-a6fb-b52843fac22a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:43.816578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c8030-68f9-4fa5-a97f-2a44b96e3391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:43.817237 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c8030-68f9-4fa5-a97f-2a44b96e3391 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:43.817530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c8030-68f9-4fa5-a97f-2a44b96e3391 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:43.817806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c8030-68f9-4fa5-a97f-2a44b96e3391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:43.819449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b41c8030-68f9-4fa5-a97f-2a44b96e3391 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:43.895026 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:43.908168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c8030-68f9-4fa5-a97f-2a44b96e3391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:43.911115 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c8030-68f9-4fa5-a97f-2a44b96e3391 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:43.928046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 717/1425] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:44.798923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bd3cc-07b6-4166-b88d-7dbdfcf4eb95 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:44.799441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bd3cc-07b6-4166-b88d-7dbdfcf4eb95 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:44.799441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bd3cc-07b6-4166-b88d-7dbdfcf4eb95 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:44.800779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:44.806339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e78dd4-5851-444c-b597-a559212b4e30 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:44.806771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e78dd4-5851-444c-b597-a559212b4e30 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:44.806879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e78dd4-5851-444c-b597-a559212b4e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:44.884000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-26e78dd4-5851-444c-b597-a559212b4e30 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:44.885271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-26e78dd4-5851-444c-b597-a559212b4e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:44.981514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:45.131359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0bd3cc-07b6-4166-b88d-7dbdfcf4eb95 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:45.134338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cf0bd3cc-07b6-4166-b88d-7dbdfcf4eb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:45.217618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:45.279718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0bd3cc-07b6-4166-b88d-7dbdfcf4eb95 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:45.400841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932badc7-0351-4642-ba02-85de98b7e10a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:45.401631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932badc7-0351-4642-ba02-85de98b7e10a service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:45.402035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932badc7-0351-4642-ba02-85de98b7e10a service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:45.402440 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932badc7-0351-4642-ba02-85de98b7e10a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:45.404853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-932badc7-0351-4642-ba02-85de98b7e10a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:45.452714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:45.493933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932badc7-0351-4642-ba02-85de98b7e10a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:45.495529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932badc7-0351-4642-ba02-85de98b7e10a service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:45.513990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 710/1428] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:46.387816 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5f6a4-cd80-4abe-aa3e-e89f738264be None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:46.388377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5f6a4-cd80-4abe-aa3e-e89f738264be None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:46.388377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5f6a4-cd80-4abe-aa3e-e89f738264be None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:46.389825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:46.394711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326cf58-bef8-46e0-bceb-3e6aa60ff148 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:46.395148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326cf58-bef8-46e0-bceb-3e6aa60ff148 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:46.395277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326cf58-bef8-46e0-bceb-3e6aa60ff148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:46.474508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1326cf58-bef8-46e0-bceb-3e6aa60ff148 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:46.475828 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1326cf58-bef8-46e0-bceb-3e6aa60ff148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:46.573462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:46.723628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5f6a4-cd80-4abe-aa3e-e89f738264be None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:46.725398 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-06f5f6a4-cd80-4abe-aa3e-e89f738264be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:46.774703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:46.871575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5f6a4-cd80-4abe-aa3e-e89f738264be None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:46.982760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d9007-802e-4ff4-9d48-98f5fca8c040 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:46.983641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d9007-802e-4ff4-9d48-98f5fca8c040 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:46.983926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d9007-802e-4ff4-9d48-98f5fca8c040 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:46.984204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d9007-802e-4ff4-9d48-98f5fca8c040 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:46.985874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bd8d9007-802e-4ff4-9d48-98f5fca8c040 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:46.992685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:47.074929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d9007-802e-4ff4-9d48-98f5fca8c040 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:47.076468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d9007-802e-4ff4-9d48-98f5fca8c040 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:47.094760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 720/1431] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:47.976138 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ed4ab-a843-42e4-9614-a4a2c3dc049f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:47.976693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ed4ab-a843-42e4-9614-a4a2c3dc049f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:47.976693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ed4ab-a843-42e4-9614-a4a2c3dc049f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:47.978124 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:47.983566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d106dc3-ccd9-4cf2-acc5-42dd6f775d46 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:47.983985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d106dc3-ccd9-4cf2-acc5-42dd6f775d46 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:47.984146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d106dc3-ccd9-4cf2-acc5-42dd6f775d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:48.063484 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1d106dc3-ccd9-4cf2-acc5-42dd6f775d46 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:48.064694 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1d106dc3-ccd9-4cf2-acc5-42dd6f775d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:48.141531 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:48.171344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:48.323348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ed4ab-a843-42e4-9614-a4a2c3dc049f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:48.324891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1b9ed4ab-a843-42e4-9614-a4a2c3dc049f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:48.343021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:48.475803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ed4ab-a843-42e4-9614-a4a2c3dc049f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:48.563547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:48.589735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad2e8-0a94-4b97-bcab-b333e40f88b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:48.590581 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad2e8-0a94-4b97-bcab-b333e40f88b5 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:48.590925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad2e8-0a94-4b97-bcab-b333e40f88b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:48.591352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad2e8-0a94-4b97-bcab-b333e40f88b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:48.593399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-214ad2e8-0a94-4b97-bcab-b333e40f88b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:48.808522 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ad2e8-0a94-4b97-bcab-b333e40f88b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:48.810066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ad2e8-0a94-4b97-bcab-b333e40f88b5 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:48.828074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 713/1434] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:49.700958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32d5b9-3a39-4b6d-9df5-593f418dca26 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:49.701512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32d5b9-3a39-4b6d-9df5-593f418dca26 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:49.701512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32d5b9-3a39-4b6d-9df5-593f418dca26 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:49.702831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:49.707800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd28dd-8015-43ab-835f-48855a00376a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:49.708263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd28dd-8015-43ab-835f-48855a00376a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:49.708437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd28dd-8015-43ab-835f-48855a00376a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:49.792586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-eddd28dd-8015-43ab-835f-48855a00376a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:49.793858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-eddd28dd-8015-43ab-835f-48855a00376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:49.838174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:49.892634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:50.044388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32d5b9-3a39-4b6d-9df5-593f418dca26 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:50.045872 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4c32d5b9-3a39-4b6d-9df5-593f418dca26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:50.191008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32d5b9-3a39-4b6d-9df5-593f418dca26 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:50.248110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:50.308440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548fbf5-863c-4f65-b185-ede250f6cb61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:50.309065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548fbf5-863c-4f65-b185-ede250f6cb61 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:50.309350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548fbf5-863c-4f65-b185-ede250f6cb61 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:50.309622 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548fbf5-863c-4f65-b185-ede250f6cb61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:50.311313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5548fbf5-863c-4f65-b185-ede250f6cb61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:50.408926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548fbf5-863c-4f65-b185-ede250f6cb61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:50.410568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548fbf5-863c-4f65-b185-ede250f6cb61 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:50.436562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 723/1437] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:51.375477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1121450-89da-4b2a-a342-de5b344cf326 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:51.376077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1121450-89da-4b2a-a342-de5b344cf326 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:51.376077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1121450-89da-4b2a-a342-de5b344cf326 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:51.377458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:51.383026 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c8046-b650-4154-81c3-1c91b168ed6c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:51.383491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c8046-b650-4154-81c3-1c91b168ed6c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:51.383621 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c8046-b650-4154-81c3-1c91b168ed6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:51.585774 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-fe9c8046-b650-4154-81c3-1c91b168ed6c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:51.587020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fe9c8046-b650-4154-81c3-1c91b168ed6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:51.589223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:51.683123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:51.836430 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1121450-89da-4b2a-a342-de5b344cf326 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:51.838013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c1121450-89da-4b2a-a342-de5b344cf326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:51.982796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1121450-89da-4b2a-a342-de5b344cf326 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:52.010468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:52.098416 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65976b1e-0fea-4817-bdea-dea29915a566 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:52.099035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65976b1e-0fea-4817-bdea-dea29915a566 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:52.099320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65976b1e-0fea-4817-bdea-dea29915a566 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:52.099597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65976b1e-0fea-4817-bdea-dea29915a566 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:52.101799 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-65976b1e-0fea-4817-bdea-dea29915a566 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:52.189936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65976b1e-0fea-4817-bdea-dea29915a566 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:52.191526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65976b1e-0fea-4817-bdea-dea29915a566 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:52.209404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 716/1440] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:54.115712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7db38-4a4b-4d9f-9910-111bd8251862 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.116767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7db38-4a4b-4d9f-9910-111bd8251862 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.116767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7db38-4a4b-4d9f-9910-111bd8251862 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.118170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:54.123314 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e35fee-0050-4b09-82a2-9eb9a0debc0a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.123732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e35fee-0050-4b09-82a2-9eb9a0debc0a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.123874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e35fee-0050-4b09-82a2-9eb9a0debc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.166534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:54.202669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-92e35fee-0050-4b09-82a2-9eb9a0debc0a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:54.203903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-92e35fee-0050-4b09-82a2-9eb9a0debc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:54.304501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:54.342937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7db38-4a4b-4d9f-9910-111bd8251862 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:54.344521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-09b7db38-4a4b-4d9f-9910-111bd8251862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:54.447355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7db38-4a4b-4d9f-9910-111bd8251862 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:54.480875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:54.526497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d5491-2332-4dfc-8872-4ed397518ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:54.527122 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d5491-2332-4dfc-8872-4ed397518ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:54.527405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d5491-2332-4dfc-8872-4ed397518ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:54.527684 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d5491-2332-4dfc-8872-4ed397518ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:54.529372 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-de7d5491-2332-4dfc-8872-4ed397518ba3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:54.637690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d5491-2332-4dfc-8872-4ed397518ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:54.643074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d5491-2332-4dfc-8872-4ed397518ba3 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:54.657873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 726/1443] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:56.810386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0e07-ccb2-4a96-add9-111ecf15c05e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:56.810923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0e07-ccb2-4a96-add9-111ecf15c05e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:56.810923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0e07-ccb2-4a96-add9-111ecf15c05e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:56.812266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:56.817588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d731d6-4195-4d8b-b56a-1b31be5aae47 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:56.818019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d731d6-4195-4d8b-b56a-1b31be5aae47 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:56.818127 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d731d6-4195-4d8b-b56a-1b31be5aae47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:56.884869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-58d731d6-4195-4d8b-b56a-1b31be5aae47 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:56.886118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-58d731d6-4195-4d8b-b56a-1b31be5aae47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:56.899182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:56.981522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:57.021315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d0e07-ccb2-4a96-add9-111ecf15c05e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:57.022863 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0d7d0e07-ccb2-4a96-add9-111ecf15c05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:57.126856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d0e07-ccb2-4a96-add9-111ecf15c05e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:05:57.139607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:57.207042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc42a0c-db18-44dc-b491-ac03d4f7d2d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:05:57.207669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42a0c-db18-44dc-b491-ac03d4f7d2d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:57.207956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42a0c-db18-44dc-b491-ac03d4f7d2d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:57.208237 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42a0c-db18-44dc-b491-ac03d4f7d2d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:57.209932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6cc42a0c-db18-44dc-b491-ac03d4f7d2d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:57.318114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc42a0c-db18-44dc-b491-ac03d4f7d2d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:05:57.320232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc42a0c-db18-44dc-b491-ac03d4f7d2d7 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:05:57.337924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 719/1446] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:05:59.691399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0963cae-d294-4cec-b9d9-8aee06f39f83 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:59.691973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0963cae-d294-4cec-b9d9-8aee06f39f83 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:59.691973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0963cae-d294-4cec-b9d9-8aee06f39f83 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:59.693272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:05:59.698494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9d179-f434-48f8-8b35-e32926f07b1a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:05:59.698927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9d179-f434-48f8-8b35-e32926f07b1a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:05:59.699045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9d179-f434-48f8-8b35-e32926f07b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:05:59.749512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:05:59.767535 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-85a9d179-f434-48f8-8b35-e32926f07b1a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:05:59.768742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-85a9d179-f434-48f8-8b35-e32926f07b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:05:59.864748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:05:59.903459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0963cae-d294-4cec-b9d9-8aee06f39f83 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:05:59.905065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d0963cae-d294-4cec-b9d9-8aee06f39f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:00.007537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0963cae-d294-4cec-b9d9-8aee06f39f83 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:00.047206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:00.079785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd27507-9566-4ff9-a3e6-4606cbad43ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:00.080399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd27507-9566-4ff9-a3e6-4606cbad43ac service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:00.080678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd27507-9566-4ff9-a3e6-4606cbad43ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:00.080952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd27507-9566-4ff9-a3e6-4606cbad43ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:00.082600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5bd27507-9566-4ff9-a3e6-4606cbad43ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:00.193860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd27507-9566-4ff9-a3e6-4606cbad43ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:00.195969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd27507-9566-4ff9-a3e6-4606cbad43ac service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:00.214322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 729/1449] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:02.349441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a6f58-d26e-46af-ade6-d4f722a1a039 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:02.350019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a6f58-d26e-46af-ade6-d4f722a1a039 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:02.350019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a6f58-d26e-46af-ade6-d4f722a1a039 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:02.351937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:02.357283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36017939-197d-44ef-a120-7408c4649bfb None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:02.364880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36017939-197d-44ef-a120-7408c4649bfb None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:02.364880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36017939-197d-44ef-a120-7408c4649bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:02.425387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-36017939-197d-44ef-a120-7408c4649bfb None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:02.426610 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-36017939-197d-44ef-a120-7408c4649bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:02.446714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:02.522487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:02.561081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a6f58-d26e-46af-ade6-d4f722a1a039 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:02.562617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a23a6f58-d26e-46af-ade6-d4f722a1a039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:02.666137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a6f58-d26e-46af-ade6-d4f722a1a039 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:02.686577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:02.740259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d36b6-ae0c-43f3-932f-15c18e170ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:02.740881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d36b6-ae0c-43f3-932f-15c18e170ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:02.741794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d36b6-ae0c-43f3-932f-15c18e170ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:02.742068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d36b6-ae0c-43f3-932f-15c18e170ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:02.743744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ce4d36b6-ae0c-43f3-932f-15c18e170ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:02.852560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d36b6-ae0c-43f3-932f-15c18e170ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:02.854049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d36b6-ae0c-43f3-932f-15c18e170ee4 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:02.874393 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 722/1452] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:04.923176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdafb6e-3adb-45bd-bbc1-f87dc61e74fb None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:04.923737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdafb6e-3adb-45bd-bbc1-f87dc61e74fb None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:04.923737 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdafb6e-3adb-45bd-bbc1-f87dc61e74fb None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:04.925113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:04.930819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9cb3-3829-4c10-8c09-59f338accefd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:04.931267 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9cb3-3829-4c10-8c09-59f338accefd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:04.931366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d9cb3-3829-4c10-8c09-59f338accefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:04.999170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:05.001947 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-129d9cb3-3829-4c10-8c09-59f338accefd None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:05.003190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-129d9cb3-3829-4c10-8c09-59f338accefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:05.100275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:05.138067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdafb6e-3adb-45bd-bbc1-f87dc61e74fb None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:05.139622 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4fdafb6e-3adb-45bd-bbc1-f87dc61e74fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:05.242260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdafb6e-3adb-45bd-bbc1-f87dc61e74fb None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:05.291105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:05.316831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7121f-c0c0-48ba-9677-53532698e03c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:05.317498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7121f-c0c0-48ba-9677-53532698e03c service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:05.317801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7121f-c0c0-48ba-9677-53532698e03c service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:05.318076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7121f-c0c0-48ba-9677-53532698e03c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:05.319775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6bb7121f-c0c0-48ba-9677-53532698e03c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:05.428191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7121f-c0c0-48ba-9677-53532698e03c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:05.430015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7121f-c0c0-48ba-9677-53532698e03c service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:05.448111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 732/1455] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:07.651213 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8645b-b8b6-4939-9fda-91577935dd02 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:07.651795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8645b-b8b6-4939-9fda-91577935dd02 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:07.651795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8645b-b8b6-4939-9fda-91577935dd02 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:07.653178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:07.658489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948edbb-49fd-4050-8e53-85639a20858b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:07.658880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948edbb-49fd-4050-8e53-85639a20858b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:07.659038 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948edbb-49fd-4050-8e53-85639a20858b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:07.724123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6948edbb-49fd-4050-8e53-85639a20858b None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:07.725968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6948edbb-49fd-4050-8e53-85639a20858b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:07.755062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:07.822285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:07.861202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8645b-b8b6-4939-9fda-91577935dd02 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:07.862735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fca8645b-b8b6-4939-9fda-91577935dd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:07.966106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8645b-b8b6-4939-9fda-91577935dd02 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:07.994266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:08.040387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cb2fe-bc7e-462f-9c2a-e7d903346ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:08.041057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cb2fe-bc7e-462f-9c2a-e7d903346ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:08.041365 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cb2fe-bc7e-462f-9c2a-e7d903346ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:08.041641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cb2fe-bc7e-462f-9c2a-e7d903346ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:08.043270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f25cb2fe-bc7e-462f-9c2a-e7d903346ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:08.150081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25cb2fe-bc7e-462f-9c2a-e7d903346ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:08.151519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25cb2fe-bc7e-462f-9c2a-e7d903346ee3 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:08.169727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 725/1458] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:10.224317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f2006-fe7b-4764-8da7-ca196efd6eb7 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:10.224878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f2006-fe7b-4764-8da7-ca196efd6eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:10.224878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f2006-fe7b-4764-8da7-ca196efd6eb7 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:10.226823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:10.232316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d918-de34-4654-8fe6-70a9180e9309 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:10.236740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d918-de34-4654-8fe6-70a9180e9309 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:10.236740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d918-de34-4654-8fe6-70a9180e9309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:10.305605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-4321d918-de34-4654-8fe6-70a9180e9309 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:10.306831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4321d918-de34-4654-8fe6-70a9180e9309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:10.314997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:10.404632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:10.441581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f2006-fe7b-4764-8da7-ca196efd6eb7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:10.443144 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-197f2006-fe7b-4764-8da7-ca196efd6eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:10.543768 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f2006-fe7b-4764-8da7-ca196efd6eb7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:10.598716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:10.617488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf5cd4-085e-4d6c-a917-b6e43899241a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:10.619322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf5cd4-085e-4d6c-a917-b6e43899241a service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:10.619322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf5cd4-085e-4d6c-a917-b6e43899241a service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:10.619322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf5cd4-085e-4d6c-a917-b6e43899241a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:10.620454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7dcf5cd4-085e-4d6c-a917-b6e43899241a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:10.734830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf5cd4-085e-4d6c-a917-b6e43899241a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:10.736236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf5cd4-085e-4d6c-a917-b6e43899241a service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:10.754592 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 735/1461] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:12.789791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330cc94-3d24-4fe9-a627-70892efbee33 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:12.789791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330cc94-3d24-4fe9-a627-70892efbee33 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:12.789791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330cc94-3d24-4fe9-a627-70892efbee33 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:12.790685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:06:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:12.796007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d72fb0-17c2-45b2-9f5a-b5ce51134a6e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:12.796405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d72fb0-17c2-45b2-9f5a-b5ce51134a6e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:12.796554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d72fb0-17c2-45b2-9f5a-b5ce51134a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:12.861901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-09d72fb0-17c2-45b2-9f5a-b5ce51134a6e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:12.863104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-09d72fb0-17c2-45b2-9f5a-b5ce51134a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:12.900544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:12.960506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:12.999832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330cc94-3d24-4fe9-a627-70892efbee33 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:13.001462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8330cc94-3d24-4fe9-a627-70892efbee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:13.118764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330cc94-3d24-4fe9-a627-70892efbee33 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:13.155434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:13.192590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3208be-c1ff-4c14-9b08-30f2318e1d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:13.193245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3208be-c1ff-4c14-9b08-30f2318e1d34 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:13.193530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3208be-c1ff-4c14-9b08-30f2318e1d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:13.193802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3208be-c1ff-4c14-9b08-30f2318e1d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:13.195451 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2d3208be-c1ff-4c14-9b08-30f2318e1d34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:13.302411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3208be-c1ff-4c14-9b08-30f2318e1d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:13.304394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3208be-c1ff-4c14-9b08-30f2318e1d34 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:13.323520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 728/1464] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:15.333617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3cb8f-4154-4e6a-ab65-a8268e50c5b5 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:15.334200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3cb8f-4154-4e6a-ab65-a8268e50c5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:15.334200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3cb8f-4154-4e6a-ab65-a8268e50c5b5 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:15.335596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:15.340843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d5208-a288-4bcd-8cc6-d9ea86123fe3 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:15.341320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d5208-a288-4bcd-8cc6-d9ea86123fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:15.341431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d5208-a288-4bcd-8cc6-d9ea86123fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:15.407983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-549d5208-a288-4bcd-8cc6-d9ea86123fe3 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:15.409269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-549d5208-a288-4bcd-8cc6-d9ea86123fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:15.428569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:15.507871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:06:15.544689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3cb8f-4154-4e6a-ab65-a8268e50c5b5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:15.546280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f1e3cb8f-4154-4e6a-ab65-a8268e50c5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:15.649403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3cb8f-4154-4e6a-ab65-a8268e50c5b5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:15.715125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:15.721913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df16f6-593d-494b-a3ea-800d13b6f176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:15.725434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df16f6-593d-494b-a3ea-800d13b6f176 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:15.725434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df16f6-593d-494b-a3ea-800d13b6f176 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:15.725434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df16f6-593d-494b-a3ea-800d13b6f176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:15.725434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-18df16f6-593d-494b-a3ea-800d13b6f176 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:15.832831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df16f6-593d-494b-a3ea-800d13b6f176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:15.834793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df16f6-593d-494b-a3ea-800d13b6f176 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:15.853177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 738/1467] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:17.917158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c1c4-6507-4939-ac7c-26ff2ce1838d None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:17.917715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c1c4-6507-4939-ac7c-26ff2ce1838d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:17.917715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c1c4-6507-4939-ac7c-26ff2ce1838d None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:17.919104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:17.924302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7115ff-0d88-40be-967f-486b9e780362 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:17.924746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7115ff-0d88-40be-967f-486b9e780362 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:17.924857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7115ff-0d88-40be-967f-486b9e780362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:17.990164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7a7115ff-0d88-40be-967f-486b9e780362 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:17.991442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7a7115ff-0d88-40be-967f-486b9e780362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:18.038496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:18.085951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:06:18.122890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c1c4-6507-4939-ac7c-26ff2ce1838d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:18.124455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1458c1c4-6507-4939-ac7c-26ff2ce1838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:18.240074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c1c4-6507-4939-ac7c-26ff2ce1838d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:18.286735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:18.312686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82d376-e15d-4630-b9f1-147e2649159d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:18.313327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82d376-e15d-4630-b9f1-147e2649159d service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:18.313610 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82d376-e15d-4630-b9f1-147e2649159d service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:18.313883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82d376-e15d-4630-b9f1-147e2649159d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:18.315508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1e82d376-e15d-4630-b9f1-147e2649159d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:18.427127 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82d376-e15d-4630-b9f1-147e2649159d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:18.428619 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82d376-e15d-4630-b9f1-147e2649159d service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:18.448933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 731/1470] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:28.084207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=22 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:28.445418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=66 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:38.805381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df9c20-a6da-4155-a9e9-82c04373e4dc None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:38.805900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df9c20-a6da-4155-a9e9-82c04373e4dc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:38.805900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df9c20-a6da-4155-a9e9-82c04373e4dc None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:38.807221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 740/1471] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:38.812465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564e068-ebf1-41dc-8898-b469f9693c49 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:38.812869 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564e068-ebf1-41dc-8898-b469f9693c49 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:38.813048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564e068-ebf1-41dc-8898-b469f9693c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:38.879079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3564e068-ebf1-41dc-8898-b469f9693c49 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:38.880298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3564e068-ebf1-41dc-8898-b469f9693c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:38.977918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 732/1472] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:39.060525 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1412ad-2f33-48f2-bef0-e27e307aaee6 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:39.060811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1412ad-2f33-48f2-bef0-e27e307aaee6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:39.061129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1412ad-2f33-48f2-bef0-e27e307aaee6 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:39.062482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 741/1473] 10.222.0.86 () {54 vars in 966 bytes} [Thu Jul 28 18:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 18:06:39.067297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5220e-309b-4d08-b483-abc11ffc8582 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:39.067584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5220e-309b-4d08-b483-abc11ffc8582 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:39.067917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5220e-309b-4d08-b483-abc11ffc8582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:39.142955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9fa5220e-309b-4d08-b483-abc11ffc8582 None None] MFA Rules not processed for user `c47f1552e73648ceb8a421c0543c2974`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:39.144174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9fa5220e-309b-4d08-b483-abc11ffc8582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:39.165580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:39.220558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 733/1474] 10.222.0.86 () {58 vars in 1063 bytes} [Thu Jul 28 18:06:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:39.228795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1412ad-2f33-48f2-bef0-e27e307aaee6 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:39.230365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ca1412ad-2f33-48f2-bef0-e27e307aaee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:39.333753 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1412ad-2f33-48f2-bef0-e27e307aaee6 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:39.406266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9008619-774e-4293-ba00-c7ea8f90ba55 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47f1552e73648ceb8a421c0543c2974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:39.406885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008619-774e-4293-ba00-c7ea8f90ba55 service glance] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:39.407167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008619-774e-4293-ba00-c7ea8f90ba55 service glance] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:39.407444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008619-774e-4293-ba00-c7ea8f90ba55 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:39.409173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f9008619-774e-4293-ba00-c7ea8f90ba55 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:39.464698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:39.519803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9008619-774e-4293-ba00-c7ea8f90ba55 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:39.521906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9008619-774e-4293-ba00-c7ea8f90ba55 service glance] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:39.540024 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 742/1475] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:41.629822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ed3f8-cf1f-45b1-a1d6-a111a39ad6bd None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:41.630413 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ed3f8-cf1f-45b1-a1d6-a111a39ad6bd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:41.630543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ed3f8-cf1f-45b1-a1d6-a111a39ad6bd None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:41.631981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 734/1476] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:41.637116 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154975c-5fad-4250-a3f0-5cb4c0fdea8d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:41.637577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154975c-5fad-4250-a3f0-5cb4c0fdea8d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:41.637689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154975c-5fad-4250-a3f0-5cb4c0fdea8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:41.702358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d154975c-5fad-4250-a3f0-5cb4c0fdea8d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:41.703501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d154975c-5fad-4250-a3f0-5cb4c0fdea8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:41.800710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 743/1477] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:41.824121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ed3f8-cf1f-45b1-a1d6-a111a39ad6bd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:41.825714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fe8ed3f8-cf1f-45b1-a1d6-a111a39ad6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:41.949963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ed3f8-cf1f-45b1-a1d6-a111a39ad6bd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:41.998246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:42.040956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd88ee-3156-43e7-92a8-0a9a1f7aa573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:42.041599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd88ee-3156-43e7-92a8-0a9a1f7aa573 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:42.041884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd88ee-3156-43e7-92a8-0a9a1f7aa573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:42.042157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd88ee-3156-43e7-92a8-0a9a1f7aa573 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:42.042886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd88ee-3156-43e7-92a8-0a9a1f7aa573 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:42.044611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-c2bd88ee-3156-43e7-92a8-0a9a1f7aa573 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:06:42.044611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: warnings.warn(msg) Jul 28 18:06:42.044611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: Jul 28 18:06:42.045789 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd88ee-3156-43e7-92a8-0a9a1f7aa573 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:42.064891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 735/1478] 10.222.0.86 () {56 vars in 1108 bytes} [Thu Jul 28 18:06:41 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:06:44.154249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de38584-15a1-47c1-8abd-c0c23af565fa None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:44.154835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de38584-15a1-47c1-8abd-c0c23af565fa None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:44.154835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de38584-15a1-47c1-8abd-c0c23af565fa None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:44.156215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 744/1479] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:44.162240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab54025-2444-4640-a51c-cc237c6d179e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:44.162687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab54025-2444-4640-a51c-cc237c6d179e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:44.162808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab54025-2444-4640-a51c-cc237c6d179e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:44.229952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5ab54025-2444-4640-a51c-cc237c6d179e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:44.231175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5ab54025-2444-4640-a51c-cc237c6d179e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:44.330016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 736/1480] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:44.373992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de38584-15a1-47c1-8abd-c0c23af565fa None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:44.375524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3de38584-15a1-47c1-8abd-c0c23af565fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:44.387342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:44.538124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de38584-15a1-47c1-8abd-c0c23af565fa None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:44.618917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:44.651046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797253c-3bf6-440f-a023-12520d162093 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:44.651665 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797253c-3bf6-440f-a023-12520d162093 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:44.652048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797253c-3bf6-440f-a023-12520d162093 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:44.652326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797253c-3bf6-440f-a023-12520d162093 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:44.654016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b797253c-3bf6-440f-a023-12520d162093 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:44.747883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797253c-3bf6-440f-a023-12520d162093 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:44.749417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797253c-3bf6-440f-a023-12520d162093 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:44.768100 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 745/1481] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:46.952992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19789c-8ffc-4389-9440-5e7900e23400 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:46.953552 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19789c-8ffc-4389-9440-5e7900e23400 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:46.953552 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19789c-8ffc-4389-9440-5e7900e23400 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:46.954975 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 737/1482] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:46.960209 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a586a4c-0237-4766-9870-17e20e414b2d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:46.960617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a586a4c-0237-4766-9870-17e20e414b2d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:46.960761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a586a4c-0237-4766-9870-17e20e414b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:47.027316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6a586a4c-0237-4766-9870-17e20e414b2d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:47.028524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6a586a4c-0237-4766-9870-17e20e414b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:47.072699 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:47.122047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 746/1483] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:47.157696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19789c-8ffc-4389-9440-5e7900e23400 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:47.159265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6b19789c-8ffc-4389-9440-5e7900e23400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:47.183540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:47.263957 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19789c-8ffc-4389-9440-5e7900e23400 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:47.338753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb665692-d1f1-464b-8b35-777adfe46834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:47.339371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb665692-d1f1-464b-8b35-777adfe46834 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:47.339658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb665692-d1f1-464b-8b35-777adfe46834 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:47.339937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb665692-d1f1-464b-8b35-777adfe46834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:47.341710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cb665692-d1f1-464b-8b35-777adfe46834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:47.416079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:47.451609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb665692-d1f1-464b-8b35-777adfe46834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:47.453215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb665692-d1f1-464b-8b35-777adfe46834 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:47.472241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 738/1484] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:49.625548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0bd02-08cd-46ce-9124-7a0ba04c32af None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:49.626087 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0bd02-08cd-46ce-9124-7a0ba04c32af None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:49.626087 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0bd02-08cd-46ce-9124-7a0ba04c32af None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:49.627476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 747/1485] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:49.632807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d92cb-59ca-4ebf-a635-c0dc9429b2f2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:49.633269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d92cb-59ca-4ebf-a635-c0dc9429b2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:49.633380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d92cb-59ca-4ebf-a635-c0dc9429b2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:49.705500 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7c4d92cb-59ca-4ebf-a635-c0dc9429b2f2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:49.706624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7c4d92cb-59ca-4ebf-a635-c0dc9429b2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:49.807068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 739/1486] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:49.845445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0bd02-08cd-46ce-9124-7a0ba04c32af None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:49.846990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-15a0bd02-08cd-46ce-9124-7a0ba04c32af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:49.949618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0bd02-08cd-46ce-9124-7a0ba04c32af None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:50.020156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:50.021876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe77468-4619-4754-a7f8-d49ef96502ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:50.022491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe77468-4619-4754-a7f8-d49ef96502ff service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:50.022766 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe77468-4619-4754-a7f8-d49ef96502ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:50.023034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe77468-4619-4754-a7f8-d49ef96502ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:50.024732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2fe77468-4619-4754-a7f8-d49ef96502ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:50.133244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe77468-4619-4754-a7f8-d49ef96502ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:50.134721 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe77468-4619-4754-a7f8-d49ef96502ff service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:50.153697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 748/1487] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:06:52.316697 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670a31-fae3-4c98-b124-1c67ea24262b None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:52.317272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670a31-fae3-4c98-b124-1c67ea24262b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:52.317272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670a31-fae3-4c98-b124-1c67ea24262b None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:52.318670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 740/1488] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:52.324136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895e3fc-11bd-4474-ae52-c4ba6e7c4cce None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:52.324568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895e3fc-11bd-4474-ae52-c4ba6e7c4cce None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:52.324695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895e3fc-11bd-4474-ae52-c4ba6e7c4cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:52.394376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0895e3fc-11bd-4474-ae52-c4ba6e7c4cce None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:52.395597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0895e3fc-11bd-4474-ae52-c4ba6e7c4cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:52.449800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:52.495661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 749/1489] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:52.537508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9670a31-fae3-4c98-b124-1c67ea24262b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:52.539051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a9670a31-fae3-4c98-b124-1c67ea24262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:52.574157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:52.642433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9670a31-fae3-4c98-b124-1c67ea24262b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:52.715067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5dee35-816a-425c-820a-0d416f5bdfbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:52.715683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dee35-816a-425c-820a-0d416f5bdfbb service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:52.715967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dee35-816a-425c-820a-0d416f5bdfbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:52.716234 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dee35-816a-425c-820a-0d416f5bdfbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:52.718037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1d5dee35-816a-425c-820a-0d416f5bdfbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:52.802375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:52.830572 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5dee35-816a-425c-820a-0d416f5bdfbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:52.832031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5dee35-816a-425c-820a-0d416f5bdfbb service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:52.851454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 741/1490] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:06:55.255525 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd1c2b-4725-4978-8d5e-3d0e11a78783 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:55.256093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd1c2b-4725-4978-8d5e-3d0e11a78783 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:55.256093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd1c2b-4725-4978-8d5e-3d0e11a78783 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:55.257479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 750/1491] 10.222.0.86 () {54 vars in 926 bytes} [Thu Jul 28 18:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:06:55.262732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f2490-973a-46ed-af2d-752d4cbb3aa4 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:55.263143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f2490-973a-46ed-af2d-752d4cbb3aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:55.263327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f2490-973a-46ed-af2d-752d4cbb3aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:55.329834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-ad8f2490-973a-46ed-af2d-752d4cbb3aa4 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:06:55.331050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ad8f2490-973a-46ed-af2d-752d4cbb3aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:55.433052 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 742/1492] 10.222.0.86 () {58 vars in 1023 bytes} [Thu Jul 28 18:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:06:55.473730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd1c2b-4725-4978-8d5e-3d0e11a78783 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:06:55.475254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-23dd1c2b-4725-4978-8d5e-3d0e11a78783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:55.579483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd1c2b-4725-4978-8d5e-3d0e11a78783 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:06:55.651058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684df71-6364-4e4b-9016-7531c1f761e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:06:55.651675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684df71-6364-4e4b-9016-7531c1f761e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:06:55.651956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684df71-6364-4e4b-9016-7531c1f761e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:06:55.652235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684df71-6364-4e4b-9016-7531c1f761e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:06:55.653889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d684df71-6364-4e4b-9016-7531c1f761e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:06:55.662791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:06:55.763125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684df71-6364-4e4b-9016-7531c1f761e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:06:55.764540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684df71-6364-4e4b-9016-7531c1f761e7 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:06:55.782622 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 751/1493] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:07:05.431198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=82 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:07:05.780425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=50 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:32.754593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a6ed7-4fba-46ef-b306-6ab38fd69ef5 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:32.755155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a6ed7-4fba-46ef-b306-6ab38fd69ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:32.755155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a6ed7-4fba-46ef-b306-6ab38fd69ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:32.830756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-490a6ed7-4fba-46ef-b306-6ab38fd69ef5 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:32.831979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-490a6ed7-4fba-46ef-b306-6ab38fd69ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:32.927508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 743/1494] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:08:32.935689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687601dd-9914-4168-bff4-34000e4c0562 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:32.936184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687601dd-9914-4168-bff4-34000e4c0562 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:32.936287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687601dd-9914-4168-bff4-34000e4c0562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:33.012271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-687601dd-9914-4168-bff4-34000e4c0562 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:33.013460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-687601dd-9914-4168-bff4-34000e4c0562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:33.110623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 752/1495] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:08:33.115426 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a6ed7-4fba-46ef-b306-6ab38fd69ef5 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:33.116640 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-490a6ed7-4fba-46ef-b306-6ab38fd69ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:33.129502 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a6ed7-4fba-46ef-b306-6ab38fd69ef5 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:33.191409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:33.224501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9782c2e-eb8f-4872-830a-74918d5857bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:33.225155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782c2e-eb8f-4872-830a-74918d5857bc admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:33.225441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782c2e-eb8f-4872-830a-74918d5857bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:33.225713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782c2e-eb8f-4872-830a-74918d5857bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:33.226411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9782c2e-eb8f-4872-830a-74918d5857bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:08:33.226715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9782c2e-eb8f-4872-830a-74918d5857bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:33.227722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-c9782c2e-eb8f-4872-830a-74918d5857bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:08:33.227722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: warnings.warn(msg) Jul 28 18:08:33.227722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: Jul 28 18:08:33.228880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9782c2e-eb8f-4872-830a-74918d5857bc admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:33.240604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 744/1496] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:08:33.249020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687601dd-9914-4168-bff4-34000e4c0562 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:33.259158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687601dd-9914-4168-bff4-34000e4c0562 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:33.322718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:33.356678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b960bb1-da30-4886-87ff-07f55526769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:33.357331 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b960bb1-da30-4886-87ff-07f55526769f admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:33.357614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b960bb1-da30-4886-87ff-07f55526769f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:33.357889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b960bb1-da30-4886-87ff-07f55526769f admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:33.358564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b960bb1-da30-4886-87ff-07f55526769f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:33.360906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-4b960bb1-da30-4886-87ff-07f55526769f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:08:33.360906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 28 18:08:33.360906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: Jul 28 18:08:33.362119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b960bb1-da30-4886-87ff-07f55526769f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:33.492219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 753/1497] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:08:33 2022] POST /identity/v3/projects => generated 375 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:08:33.496390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9782c2e-eb8f-4872-830a-74918d5857bc admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:33.497989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c9782c2e-eb8f-4872-830a-74918d5857bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:33.532117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9782c2e-eb8f-4872-830a-74918d5857bc admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:33.624291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5d126-7bcc-4a31-a6d5-fa99b579fc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:33.624907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5d126-7bcc-4a31-a6d5-fa99b579fc28 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:33.625219 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5d126-7bcc-4a31-a6d5-fa99b579fc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:33.625989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5d126-7bcc-4a31-a6d5-fa99b579fc28 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:33.626645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5d126-7bcc-4a31-a6d5-fa99b579fc28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:33.627653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-6ba5d126-7bcc-4a31-a6d5-fa99b579fc28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:08:33.627653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: warnings.warn(msg) Jul 28 18:08:33.627653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: Jul 28 18:08:33.628859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5d126-7bcc-4a31-a6d5-fa99b579fc28 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:33.749460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 745/1498] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:08:33 2022] POST /identity/v3/users => generated 288 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:08:33.754179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b960bb1-da30-4886-87ff-07f55526769f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:33.755721 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4b960bb1-da30-4886-87ff-07f55526769f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:33.816604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b960bb1-da30-4886-87ff-07f55526769f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:33.903022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:33.913746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dbd9b-9de6-4014-be74-1afa13d2c603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:33.914363 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dbd9b-9de6-4014-be74-1afa13d2c603 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:33.914640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dbd9b-9de6-4014-be74-1afa13d2c603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:33.914910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dbd9b-9de6-4014-be74-1afa13d2c603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:33.915607 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98dbd9b-9de6-4014-be74-1afa13d2c603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:33.916601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-c98dbd9b-9de6-4014-be74-1afa13d2c603 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:08:33.916601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 28 18:08:33.916601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: Jul 28 18:08:33.917537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98dbd9b-9de6-4014-be74-1afa13d2c603 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:33.930395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 754/1499] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:08:33.934665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5d126-7bcc-4a31-a6d5-fa99b579fc28 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:33.936199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6ba5d126-7bcc-4a31-a6d5-fa99b579fc28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:33.948389 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5d126-7bcc-4a31-a6d5-fa99b579fc28 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:33.973327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:34.048055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600ad07-dfa9-4782-9f03-34820b6444cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:34.048781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600ad07-dfa9-4782-9f03-34820b6444cc admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:34.049094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600ad07-dfa9-4782-9f03-34820b6444cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:34.049367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600ad07-dfa9-4782-9f03-34820b6444cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:34.050103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600ad07-dfa9-4782-9f03-34820b6444cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:34.051300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-2600ad07-dfa9-4782-9f03-34820b6444cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:08:34.051300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: warnings.warn(msg) Jul 28 18:08:34.051300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: Jul 28 18:08:34.052308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600ad07-dfa9-4782-9f03-34820b6444cc admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:34.068374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 746/1500] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:08:34.072819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dbd9b-9de6-4014-be74-1afa13d2c603 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:34.074424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c98dbd9b-9de6-4014-be74-1afa13d2c603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:34.111508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dbd9b-9de6-4014-be74-1afa13d2c603 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:34.203827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95405bd5-691d-4e08-8112-b0a776e47dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:34.204550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95405bd5-691d-4e08-8112-b0a776e47dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:34.204836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95405bd5-691d-4e08-8112-b0a776e47dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:34.205144 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95405bd5-691d-4e08-8112-b0a776e47dc2 admin admin] PATH_INFO: `/v3/projects/49f22c6070e1499583557ac70081fc68/users/15cc144b083a40e4b3548ee2fc93ad58/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:34.227582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95405bd5-691d-4e08-8112-b0a776e47dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f22c6070e1499583557ac70081fc68, user_id=15cc144b083a40e4b3548ee2fc93ad58, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:34.228956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-95405bd5-691d-4e08-8112-b0a776e47dc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:08:34.228956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 28 18:08:34.228956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: Jul 28 18:08:34.230909 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95405bd5-691d-4e08-8112-b0a776e47dc2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:34.332780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 755/1501] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:08:34 2022] PUT /identity/v3/projects/49f22c6070e1499583557ac70081fc68/users/15cc144b083a40e4b3548ee2fc93ad58/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:08:34.346536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c134f32-eebe-4510-b2e9-ba2ec971fedd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:34.346825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c134f32-eebe-4510-b2e9-ba2ec971fedd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:34.347102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c134f32-eebe-4510-b2e9-ba2ec971fedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:34.408833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8c134f32-eebe-4510-b2e9-ba2ec971fedd None None] MFA Rules not processed for user `15cc144b083a40e4b3548ee2fc93ad58`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:34.410093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8c134f32-eebe-4510-b2e9-ba2ec971fedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:34.487527 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 747/1502] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:08:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:08:34.500414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f521c8-de05-4514-9f36-1dfcc80098a9 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:34.500731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f521c8-de05-4514-9f36-1dfcc80098a9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:34.501065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f521c8-de05-4514-9f36-1dfcc80098a9 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:34.502462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 756/1503] 10.222.0.86 () {54 vars in 964 bytes} [Thu Jul 28 18:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:08:34.507205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ba562-249e-4995-a3b1-dc9cb8362248 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:34.507499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ba562-249e-4995-a3b1-dc9cb8362248 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:34.507779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ba562-249e-4995-a3b1-dc9cb8362248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:34.539224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:34.582337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c55ba562-249e-4995-a3b1-dc9cb8362248 None None] MFA Rules not processed for user `9dbaf1e343474322b2567e2dfda99d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:34.583560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c55ba562-249e-4995-a3b1-dc9cb8362248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:34.661215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 748/1504] 10.222.0.86 () {58 vars in 1061 bytes} [Thu Jul 28 18:08:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:08:34.667170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f521c8-de05-4514-9f36-1dfcc80098a9 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:34.668762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-62f521c8-de05-4514-9f36-1dfcc80098a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:34.722142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:34.775834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f521c8-de05-4514-9f36-1dfcc80098a9 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:34.849530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e130b1f-ed3d-4b63-ba14-889d48c957de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:34.850147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e130b1f-ed3d-4b63-ba14-889d48c957de service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:34.850422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e130b1f-ed3d-4b63-ba14-889d48c957de service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:34.850691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e130b1f-ed3d-4b63-ba14-889d48c957de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:34.852316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2e130b1f-ed3d-4b63-ba14-889d48c957de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:34.948492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e130b1f-ed3d-4b63-ba14-889d48c957de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:34.949940 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e130b1f-ed3d-4b63-ba14-889d48c957de service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:34.953416 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:34.967652 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 757/1505] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:08:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:08:35.013741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6ee4-543b-4580-85bd-12a8ef1e0a07 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:35.014178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6ee4-543b-4580-85bd-12a8ef1e0a07 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:35.014305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6ee4-543b-4580-85bd-12a8ef1e0a07 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:35.015627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 749/1506] 10.222.0.86 () {52 vars in 830 bytes} [Thu Jul 28 18:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:08:35.019750 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e130b1f-ed3d-4b63-ba14-889d48c957de service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:35.020995 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2e130b1f-ed3d-4b63-ba14-889d48c957de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:35.119998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e130b1f-ed3d-4b63-ba14-889d48c957de service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:35.212507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:35.213181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:35.213463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:35.213738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] PATH_INFO: `/v3/users/15cc144b083a40e4b3548ee2fc93ad58` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:35.225661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15cc144b083a40e4b3548ee2fc93ad58)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:35.226687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:08:35.226687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 28 18:08:35.226687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: Jul 28 18:08:35.228164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:35.238246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:35.435623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15cc144b083a40e4b3548ee2fc93ad58'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:35.436192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15cc144b083a40e4b3548ee2fc93ad58'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:35.449193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15cc144b083a40e4b3548ee2fc93ad58'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:35.507744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15cc144b083a40e4b3548ee2fc93ad58'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:35.508183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15cc144b083a40e4b3548ee2fc93ad58'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:35.531579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 758/1507] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:08:35 2022] DELETE /identity/v3/users/15cc144b083a40e4b3548ee2fc93ad58 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:08:35.536214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c705c21-f84b-47ae-ba0d-67cf7a655774 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:35.536505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c705c21-f84b-47ae-ba0d-67cf7a655774 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:35.536788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c705c21-f84b-47ae-ba0d-67cf7a655774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:35.592959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8c705c21-f84b-47ae-ba0d-67cf7a655774 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:08:35.594711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8c705c21-f84b-47ae-ba0d-67cf7a655774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:35.670384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:35.683340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 750/1508] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:08:35.693047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:35.694259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:35.795207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42d4f2-3770-4774-bf48-7f7529457c78 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:35.849586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:35.869168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e801e-39e5-4ee0-b08e-6d52997663aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:35.869794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e801e-39e5-4ee0-b08e-6d52997663aa service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:35.870074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e801e-39e5-4ee0-b08e-6d52997663aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:35.870346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e801e-39e5-4ee0-b08e-6d52997663aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:35.871968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d58e801e-39e5-4ee0-b08e-6d52997663aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:35.984617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e801e-39e5-4ee0-b08e-6d52997663aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:35.986541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e801e-39e5-4ee0-b08e-6d52997663aa service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:36.004814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 759/1509] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:08:36.511528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c705c21-f84b-47ae-ba0d-67cf7a655774 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:08:36.515026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8c705c21-f84b-47ae-ba0d-67cf7a655774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:08:36.525584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c705c21-f84b-47ae-ba0d-67cf7a655774 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:08:36.620657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:08:36.621336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:08:36.621619 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:08:36.621892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] PATH_INFO: `/v3/projects/49f22c6070e1499583557ac70081fc68` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:08:36.632292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f22c6070e1499583557ac70081fc68)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:08:36.633349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:08:36.633349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: warnings.warn(msg) Jul 28 18:08:36.633349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: Jul 28 18:08:36.634546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:08:36.739228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f22c6070e1499583557ac70081fc68', 'request_id': 'req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:36.745386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f22c6070e1499583557ac70081fc68', 'request_id': 'req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:36.810968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f22c6070e1499583557ac70081fc68', 'request_id': 'req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:36.841747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] The token cache is being invalidate because project 49f22c6070e1499583557ac70081fc68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:08:36.842044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:36.842575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7eacb6e-fe1b-401a-af0d-aa00178dbe3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:08:36.843922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 751/1510] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:08:36 2022] DELETE /identity/v3/projects/49f22c6070e1499583557ac70081fc68 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:08:46.002538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=62 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:08:46.839626 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:10:20.914281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ab058-b355-49a7-9eca-180298877b60 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:20.914885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ab058-b355-49a7-9eca-180298877b60 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:20.914885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ab058-b355-49a7-9eca-180298877b60 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:20.916214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:10:20.921293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6debe4-23be-43ee-a594-fecc5d5834ec None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:20.921752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6debe4-23be-43ee-a594-fecc5d5834ec None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:20.921859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6debe4-23be-43ee-a594-fecc5d5834ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:21.002110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3c6debe4-23be-43ee-a594-fecc5d5834ec None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:21.003343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3c6debe4-23be-43ee-a594-fecc5d5834ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:21.113460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:21.281611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ab058-b355-49a7-9eca-180298877b60 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:21.283775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c05ab058-b355-49a7-9eca-180298877b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:21.403437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ab058-b355-49a7-9eca-180298877b60 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:21.408979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a6363-d94a-40f7-9058-44382159197a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:21.409609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a6363-d94a-40f7-9058-44382159197a service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:21.409895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a6363-d94a-40f7-9058-44382159197a service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:21.410173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a6363-d94a-40f7-9058-44382159197a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:21.418574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a6363-d94a-40f7-9058-44382159197a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:21.420164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a6363-d94a-40f7-9058-44382159197a service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:21.430392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 761/1513] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:22.256125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11689ba8-a082-4b83-9c26-6a64837a1421 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:22.256694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11689ba8-a082-4b83-9c26-6a64837a1421 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:22.256694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11689ba8-a082-4b83-9c26-6a64837a1421 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:22.258149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:10:22.262858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda2af5-a9d8-4e5d-8b12-a46fe5ac303a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:22.263255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda2af5-a9d8-4e5d-8b12-a46fe5ac303a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:22.263417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda2af5-a9d8-4e5d-8b12-a46fe5ac303a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:22.283503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ecda2af5-a9d8-4e5d-8b12-a46fe5ac303a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:22.284752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ecda2af5-a9d8-4e5d-8b12-a46fe5ac303a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:22.292642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:22.447577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11689ba8-a082-4b83-9c26-6a64837a1421 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:22.462785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11689ba8-a082-4b83-9c26-6a64837a1421 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:22.468200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef72891-d4f2-416d-9ae2-9e6cdb45ae2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:22.468801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef72891-d4f2-416d-9ae2-9e6cdb45ae2c service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:22.469120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef72891-d4f2-416d-9ae2-9e6cdb45ae2c service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:22.469396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef72891-d4f2-416d-9ae2-9e6cdb45ae2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:22.477636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef72891-d4f2-416d-9ae2-9e6cdb45ae2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:22.479207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef72891-d4f2-416d-9ae2-9e6cdb45ae2c service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:22.489057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 754/1516] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:22.934204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22660b93-12b2-43ab-a80b-d4d2a3d4cee1 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:22.934502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22660b93-12b2-43ab-a80b-d4d2a3d4cee1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:22.934785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22660b93-12b2-43ab-a80b-d4d2a3d4cee1 None None] PATH_INFO: `/v3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:22.936103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 763/1517] 10.222.0.93 () {54 vars in 925 bytes} [Thu Jul 28 18:10:22 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:10:22.942514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f5cab-6ecb-4d8b-a3eb-a1faac78a9d4 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:22.942945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f5cab-6ecb-4d8b-a3eb-a1faac78a9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:22.943232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f5cab-6ecb-4d8b-a3eb-a1faac78a9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:23.004071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-874f5cab-6ecb-4d8b-a3eb-a1faac78a9d4 None None] MFA Rules not processed for user `b20564a83953433290aa3fabf884370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:23.005326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-874f5cab-6ecb-4d8b-a3eb-a1faac78a9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:23.024137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:10:23.080189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 755/1518] 10.222.0.93 () {58 vars in 1015 bytes} [Thu Jul 28 18:10:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:32.278600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=56 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:10:32.729282 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7cf5-075a-44ba-95f3-6b50e6445cb4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:32.729574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7cf5-075a-44ba-95f3-6b50e6445cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:32.729857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7cf5-075a-44ba-95f3-6b50e6445cb4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:32.731387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 764/1519] 10.222.0.86 () {54 vars in 961 bytes} [Thu Jul 28 18:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:10:32.736453 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bbad7-7bb4-45bc-ac93-c075097fc9bf None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:32.736991 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bbad7-7bb4-45bc-ac93-c075097fc9bf None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:32.737112 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bbad7-7bb4-45bc-ac93-c075097fc9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:32.757358 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-205bbad7-7bb4-45bc-ac93-c075097fc9bf None None] MFA Rules not processed for user `b20564a83953433290aa3fabf884370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:32.758557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-205bbad7-7bb4-45bc-ac93-c075097fc9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:32.767768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 756/1520] 10.222.0.86 () {58 vars in 1058 bytes} [Thu Jul 28 18:10:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:32.776156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc7cf5-075a-44ba-95f3-6b50e6445cb4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:32.790212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc7cf5-075a-44ba-95f3-6b50e6445cb4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:32.795183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096282a-7e57-44c3-910e-73f56b470fe5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:32.795789 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096282a-7e57-44c3-910e-73f56b470fe5 service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:32.796071 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096282a-7e57-44c3-910e-73f56b470fe5 service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:32.796346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096282a-7e57-44c3-910e-73f56b470fe5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:32.806798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096282a-7e57-44c3-910e-73f56b470fe5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:32.808580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096282a-7e57-44c3-910e-73f56b470fe5 service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:32.818794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 765/1521] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:33.736849 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1824981-a3ce-44f1-b481-c3cc952a6098 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:33.740114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1824981-a3ce-44f1-b481-c3cc952a6098 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:33.740114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1824981-a3ce-44f1-b481-c3cc952a6098 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:33.740435 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:10:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 18:10:33.745379 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183bb6a-266b-422b-b0ca-671a2bc9145a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:33.745828 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183bb6a-266b-422b-b0ca-671a2bc9145a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:33.745937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183bb6a-266b-422b-b0ca-671a2bc9145a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:33.767749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e183bb6a-266b-422b-b0ca-671a2bc9145a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:33.769011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e183bb6a-266b-422b-b0ca-671a2bc9145a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:33.777082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:33.932767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1824981-a3ce-44f1-b481-c3cc952a6098 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:33.947300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1824981-a3ce-44f1-b481-c3cc952a6098 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:33.952887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959e2b0-1fe3-4c8d-9705-ab088d6b0742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:33.953519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959e2b0-1fe3-4c8d-9705-ab088d6b0742 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:33.953812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959e2b0-1fe3-4c8d-9705-ab088d6b0742 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:33.954091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959e2b0-1fe3-4c8d-9705-ab088d6b0742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:33.962241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959e2b0-1fe3-4c8d-9705-ab088d6b0742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:33.963824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959e2b0-1fe3-4c8d-9705-ab088d6b0742 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:33.973842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 758/1524] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:34.802184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32498b8-41e7-420c-969c-de137fbb1a43 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:34.802718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32498b8-41e7-420c-969c-de137fbb1a43 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:34.802718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32498b8-41e7-420c-969c-de137fbb1a43 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:34.804052 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 18:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:10:34.808899 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907e49-7d6a-418b-907b-6c79e3a57140 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:34.809380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907e49-7d6a-418b-907b-6c79e3a57140 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:34.809511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907e49-7d6a-418b-907b-6c79e3a57140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:34.829146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-78907e49-7d6a-418b-907b-6c79e3a57140 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:34.830338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-78907e49-7d6a-418b-907b-6c79e3a57140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:34.838816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 18:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:34.990088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32498b8-41e7-420c-969c-de137fbb1a43 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:35.004892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32498b8-41e7-420c-969c-de137fbb1a43 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:35.010631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c5b52-361a-47cb-9217-12d32004b9d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:35.011238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5b52-361a-47cb-9217-12d32004b9d8 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:35.011522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5b52-361a-47cb-9217-12d32004b9d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:35.011799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5b52-361a-47cb-9217-12d32004b9d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:35.020703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c5b52-361a-47cb-9217-12d32004b9d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:35.022310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c5b52-361a-47cb-9217-12d32004b9d8 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:35.031673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 768/1527] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:38.996956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2075e2-f41f-4b8f-9e09-c947ecd00386 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:38.997531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2075e2-f41f-4b8f-9e09-c947ecd00386 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:38.997531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2075e2-f41f-4b8f-9e09-c947ecd00386 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:38.998940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:10:39.004791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5a262-d574-4c96-bcbc-b9b32420fa84 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:39.005236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5a262-d574-4c96-bcbc-b9b32420fa84 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:39.005387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5a262-d574-4c96-bcbc-b9b32420fa84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:39.024915 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-86c5a262-d574-4c96-bcbc-b9b32420fa84 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:39.026153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-86c5a262-d574-4c96-bcbc-b9b32420fa84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:39.034112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:39.108014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2075e2-f41f-4b8f-9e09-c947ecd00386 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:39.109728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8a2075e2-f41f-4b8f-9e09-c947ecd00386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:39.188445 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2075e2-f41f-4b8f-9e09-c947ecd00386 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:39.192079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62771a2-a28a-4e61-900e-4d4e639bc364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:39.192691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62771a2-a28a-4e61-900e-4d4e639bc364 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:39.192984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62771a2-a28a-4e61-900e-4d4e639bc364 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:39.193275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62771a2-a28a-4e61-900e-4d4e639bc364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:39.202197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62771a2-a28a-4e61-900e-4d4e639bc364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:39.203627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62771a2-a28a-4e61-900e-4d4e639bc364 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:39.213198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 761/1530] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:41.221722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d04c-0fd3-4fb8-8d7c-64340651d1aa None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:41.222267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d04c-0fd3-4fb8-8d7c-64340651d1aa None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:41.222267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d04c-0fd3-4fb8-8d7c-64340651d1aa None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:41.223920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:10:41.229243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d238bf8-2d88-43f5-90bb-a5ef6eb3a24c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:41.229653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d238bf8-2d88-43f5-90bb-a5ef6eb3a24c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:41.229806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d238bf8-2d88-43f5-90bb-a5ef6eb3a24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:41.250145 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7d238bf8-2d88-43f5-90bb-a5ef6eb3a24c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:41.251352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7d238bf8-2d88-43f5-90bb-a5ef6eb3a24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:41.264200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:41.299605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2d04c-0fd3-4fb8-8d7c-64340651d1aa None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:41.312271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2d04c-0fd3-4fb8-8d7c-64340651d1aa None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:41.316403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c445a-a3d9-405a-9029-4ebaae68e453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:41.317028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c445a-a3d9-405a-9029-4ebaae68e453 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:41.317313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c445a-a3d9-405a-9029-4ebaae68e453 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:41.317587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c445a-a3d9-405a-9029-4ebaae68e453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:41.326431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c445a-a3d9-405a-9029-4ebaae68e453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:41.327844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c445a-a3d9-405a-9029-4ebaae68e453 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:41.337223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 771/1533] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:45.046827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4098f8-2744-40d7-8f15-276443f6a95c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:45.047369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4098f8-2744-40d7-8f15-276443f6a95c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:45.047369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4098f8-2744-40d7-8f15-276443f6a95c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:45.048688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:10:45.054342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee28db-f0d0-4616-840b-e9d0bc46b2b7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:45.054783 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee28db-f0d0-4616-840b-e9d0bc46b2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:45.054984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee28db-f0d0-4616-840b-e9d0bc46b2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:45.074522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7eee28db-f0d0-4616-840b-e9d0bc46b2b7 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:45.075732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7eee28db-f0d0-4616-840b-e9d0bc46b2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:45.083873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:45.390345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4098f8-2744-40d7-8f15-276443f6a95c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:45.392042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-eb4098f8-2744-40d7-8f15-276443f6a95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:45.459033 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:10:45.472161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4098f8-2744-40d7-8f15-276443f6a95c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:45.476069 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0c869-459a-48d0-a85c-90e27e8bcc73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:45.476686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0c869-459a-48d0-a85c-90e27e8bcc73 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:45.477061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0c869-459a-48d0-a85c-90e27e8bcc73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:45.477345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0c869-459a-48d0-a85c-90e27e8bcc73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:45.486755 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0c869-459a-48d0-a85c-90e27e8bcc73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:45.488214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0c869-459a-48d0-a85c-90e27e8bcc73 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:45.497635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 764/1536] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:47.550722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251265dd-25d6-49b5-a381-9e5ffc0a63e1 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:47.551329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251265dd-25d6-49b5-a381-9e5ffc0a63e1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:47.551329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251265dd-25d6-49b5-a381-9e5ffc0a63e1 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:47.552690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 18:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:10:47.557943 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b1c5-24b9-480b-96c0-b535dca90c44 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:47.558391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b1c5-24b9-480b-96c0-b535dca90c44 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:47.558499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b1c5-24b9-480b-96c0-b535dca90c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:47.578268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5f65b1c5-24b9-480b-96c0-b535dca90c44 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:10:47.579475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5f65b1c5-24b9-480b-96c0-b535dca90c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:47.587859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 18:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:10:47.893691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251265dd-25d6-49b5-a381-9e5ffc0a63e1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:10:47.895387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-251265dd-25d6-49b5-a381-9e5ffc0a63e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:10:47.908429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251265dd-25d6-49b5-a381-9e5ffc0a63e1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:10:47.912092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0960d-2d7e-4bb9-9245-2d33619f3976 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:10:47.912703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0960d-2d7e-4bb9-9245-2d33619f3976 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:10:47.913009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0960d-2d7e-4bb9-9245-2d33619f3976 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:10:47.913290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0960d-2d7e-4bb9-9245-2d33619f3976 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:10:47.922585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0960d-2d7e-4bb9-9245-2d33619f3976 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:10:47.924016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0960d-2d7e-4bb9-9245-2d33619f3976 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:10:47.934334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 774/1539] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:10:57.573537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=11 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:10:57.929970 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=33 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:19.625329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb81740-9ec4-42fa-a8e7-ee73bdc19701 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:19.626011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb81740-9ec4-42fa-a8e7-ee73bdc19701 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:19.626011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb81740-9ec4-42fa-a8e7-ee73bdc19701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:19.650802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9bb81740-9ec4-42fa-a8e7-ee73bdc19701 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:19.652048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9bb81740-9ec4-42fa-a8e7-ee73bdc19701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:19.663981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 766/1540] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:19.673607 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24383c-4ede-4611-ad0c-8292a1adb67d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:19.674161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24383c-4ede-4611-ad0c-8292a1adb67d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:19.674161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24383c-4ede-4611-ad0c-8292a1adb67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:19.698283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8e24383c-4ede-4611-ad0c-8292a1adb67d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:19.700516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274c18d-480f-4bd1-91b7-6372f0b65a3e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:19.700805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274c18d-480f-4bd1-91b7-6372f0b65a3e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:19.701107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274c18d-480f-4bd1-91b7-6372f0b65a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:19.702164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8e24383c-4ede-4611-ad0c-8292a1adb67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:19.738998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7274c18d-480f-4bd1-91b7-6372f0b65a3e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:19.740598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7274c18d-480f-4bd1-91b7-6372f0b65a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:19.743712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 775/1541] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:19.748191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24383c-4ede-4611-ad0c-8292a1adb67d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:19.759184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 767/1542] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 28 18:12:19.766481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24383c-4ede-4611-ad0c-8292a1adb67d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:19.768026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab9c71-0c74-46f8-8319-4e82d8072148 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:19.768321 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab9c71-0c74-46f8-8319-4e82d8072148 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:19.768600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab9c71-0c74-46f8-8319-4e82d8072148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:19.785048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ed4b1-318c-4bb3-8918-89bbfb114c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:19.790806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ed4b1-318c-4bb3-8918-89bbfb114c92 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:19.790806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ed4b1-318c-4bb3-8918-89bbfb114c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:19.790806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ed4b1-318c-4bb3-8918-89bbfb114c92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:19.790806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ed4b1-318c-4bb3-8918-89bbfb114c92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:19.790806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ed4b1-318c-4bb3-8918-89bbfb114c92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:19.791239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-794ed4b1-318c-4bb3-8918-89bbfb114c92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:12:19.791239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 28 18:12:19.791239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: Jul 28 18:12:19.791239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ed4b1-318c-4bb3-8918-89bbfb114c92 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:19.804986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-30ab9c71-0c74-46f8-8319-4e82d8072148 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:19.806508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-30ab9c71-0c74-46f8-8319-4e82d8072148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:19.819359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 768/1543] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:19.820942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2485b90-b1f4-4479-98b4-6b3d75e30fce None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:19.821493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2485b90-b1f4-4479-98b4-6b3d75e30fce None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:19.821943 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2485b90-b1f4-4479-98b4-6b3d75e30fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:19.823995 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 776/1544] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:12:19.824151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ed4b1-318c-4bb3-8918-89bbfb114c92 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:19.843854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ed4b1-318c-4bb3-8918-89bbfb114c92 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:19.857795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd34099-2169-423e-8d74-6898ab29eae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:19.858456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd34099-2169-423e-8d74-6898ab29eae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:19.858755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd34099-2169-423e-8d74-6898ab29eae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:19.859696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd34099-2169-423e-8d74-6898ab29eae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:19.860309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e2485b90-b1f4-4479-98b4-6b3d75e30fce None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:19.861146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd34099-2169-423e-8d74-6898ab29eae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:19.861464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd34099-2169-423e-8d74-6898ab29eae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:19.862271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e2485b90-b1f4-4479-98b4-6b3d75e30fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:19.863075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd34099-2169-423e-8d74-6898ab29eae2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:19.875642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 769/1545] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:19.881910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2485b90-b1f4-4479-98b4-6b3d75e30fce None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:19.889044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 777/1546] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:19.896678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a811b-28eb-4e1f-b941-3533eb9c4fba None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:19.897272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a811b-28eb-4e1f-b941-3533eb9c4fba None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:19.897779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a811b-28eb-4e1f-b941-3533eb9c4fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:19.901012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2485b90-b1f4-4479-98b4-6b3d75e30fce None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:19.906005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4334725-5ca0-4b96-9ee6-39b7df77d1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:19.906636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4334725-5ca0-4b96-9ee6-39b7df77d1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:19.906921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4334725-5ca0-4b96-9ee6-39b7df77d1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:19.907197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4334725-5ca0-4b96-9ee6-39b7df77d1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:19.907878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4334725-5ca0-4b96-9ee6-39b7df77d1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:19.909719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-b4334725-5ca0-4b96-9ee6-39b7df77d1d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:12:19.909719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: warnings.warn(msg) Jul 28 18:12:19.909719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: Jul 28 18:12:19.910754 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4334725-5ca0-4b96-9ee6-39b7df77d1d3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:19.921069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-bf5a811b-28eb-4e1f-b941-3533eb9c4fba None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:19.922543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bf5a811b-28eb-4e1f-b941-3533eb9c4fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:19.938860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 778/1547] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:19.940472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753abd6-aece-4c79-aca7-cc73b5b17fa8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:19.940947 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753abd6-aece-4c79-aca7-cc73b5b17fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:19.941772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753abd6-aece-4c79-aca7-cc73b5b17fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:19.965021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1753abd6-aece-4c79-aca7-cc73b5b17fa8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:19.966261 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1753abd6-aece-4c79-aca7-cc73b5b17fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:19.974707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 779/1548] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:19.977003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c2e24-f32a-4d3b-beab-9596cc5f308c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:19.977003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c2e24-f32a-4d3b-beab-9596cc5f308c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:19.977003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c2e24-f32a-4d3b-beab-9596cc5f308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:19.995734 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-594c2e24-f32a-4d3b-beab-9596cc5f308c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:19.996988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-594c2e24-f32a-4d3b-beab-9596cc5f308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:20.006062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 780/1549] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:20.007061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8e23-863c-4621-bece-cca176fcab30 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.007557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8e23-863c-4621-bece-cca176fcab30 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.007691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac8e23-863c-4621-bece-cca176fcab30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.019316 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 770/1550] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:19 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:20.020539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4334725-5ca0-4b96-9ee6-39b7df77d1d3 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.028077 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-43ac8e23-863c-4621-bece-cca176fcab30 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:20.029381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-43ac8e23-863c-4621-bece-cca176fcab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:20.091600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 781/1551] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:20.092660 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4334725-5ca0-4b96-9ee6-39b7df77d1d3 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.092912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9949a3-6c2e-4218-8950-8440ed7ebbf3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.093419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9949a3-6c2e-4218-8950-8440ed7ebbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.093770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9949a3-6c2e-4218-8950-8440ed7ebbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.099985 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742c850-76bf-4c21-8ebe-4ecb6d4c1e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.100919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742c850-76bf-4c21-8ebe-4ecb6d4c1e3b admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.101386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742c850-76bf-4c21-8ebe-4ecb6d4c1e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.101805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742c850-76bf-4c21-8ebe-4ecb6d4c1e3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.102840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742c850-76bf-4c21-8ebe-4ecb6d4c1e3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.105964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742c850-76bf-4c21-8ebe-4ecb6d4c1e3b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.113942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9f9949a3-6c2e-4218-8950-8440ed7ebbf3 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:20.115159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9f9949a3-6c2e-4218-8950-8440ed7ebbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:20.123409 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 782/1552] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:20.125277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4488c-33d8-4522-b29c-d07646137947 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.125568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4488c-33d8-4522-b29c-d07646137947 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.125898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4488c-33d8-4522-b29c-d07646137947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.145224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1fe4488c-33d8-4522-b29c-d07646137947 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:20.146431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1fe4488c-33d8-4522-b29c-d07646137947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:20.154823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 783/1553] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:20.155616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4488c-33d8-4522-b29c-d07646137947 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.166541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4488c-33d8-4522-b29c-d07646137947 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.171162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f389a66-5c35-48d1-af3d-34786aeff048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.171795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f389a66-5c35-48d1-af3d-34786aeff048 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.172081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f389a66-5c35-48d1-af3d-34786aeff048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.172364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f389a66-5c35-48d1-af3d-34786aeff048 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.173112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f389a66-5c35-48d1-af3d-34786aeff048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:20.173412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f389a66-5c35-48d1-af3d-34786aeff048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.175009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f389a66-5c35-48d1-af3d-34786aeff048 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.186947 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 784/1554] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:20.188123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2c1dc-aa53-41fb-9588-fd741db9ac48 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.188415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2c1dc-aa53-41fb-9588-fd741db9ac48 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.188695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2c1dc-aa53-41fb-9588-fd741db9ac48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.210962 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b9b2c1dc-aa53-41fb-9588-fd741db9ac48 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:20.212205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b9b2c1dc-aa53-41fb-9588-fd741db9ac48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:20.221459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 785/1555] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:20.224322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2c1dc-aa53-41fb-9588-fd741db9ac48 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.225956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 771/1556] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/projects => generated 355 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:20.228631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea6bef-0361-425a-bbdf-e6c266f9e2d8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.229895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea6bef-0361-425a-bbdf-e6c266f9e2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.230338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea6bef-0361-425a-bbdf-e6c266f9e2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.233214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2c1dc-aa53-41fb-9588-fd741db9ac48 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.253240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-65ea6bef-0361-425a-bbdf-e6c266f9e2d8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:20.254536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-65ea6bef-0361-425a-bbdf-e6c266f9e2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:20.289670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2dac5-6c6c-4b18-8c53-583711182c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.290576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2dac5-6c6c-4b18-8c53-583711182c6c admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.290858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2dac5-6c6c-4b18-8c53-583711182c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.291135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2dac5-6c6c-4b18-8c53-583711182c6c admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.291806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2dac5-6c6c-4b18-8c53-583711182c6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.292819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-45e2dac5-6c6c-4b18-8c53-583711182c6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:12:20.292819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 28 18:12:20.292819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: Jul 28 18:12:20.294059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2dac5-6c6c-4b18-8c53-583711182c6c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.315342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 772/1557] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:20.316600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f602877-0c43-46f1-a7ca-56a42fe78d6b None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.316895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f602877-0c43-46f1-a7ca-56a42fe78d6b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.317205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f602877-0c43-46f1-a7ca-56a42fe78d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.336848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1f602877-0c43-46f1-a7ca-56a42fe78d6b None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:20.338081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1f602877-0c43-46f1-a7ca-56a42fe78d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:20.346931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 773/1558] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:20.348529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f602877-0c43-46f1-a7ca-56a42fe78d6b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.356679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f602877-0c43-46f1-a7ca-56a42fe78d6b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.361412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca85b1-4ebf-436c-abaf-7edf471bac25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.361991 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca85b1-4ebf-436c-abaf-7edf471bac25 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.362279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca85b1-4ebf-436c-abaf-7edf471bac25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.362560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca85b1-4ebf-436c-abaf-7edf471bac25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.363248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca85b1-4ebf-436c-abaf-7edf471bac25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:20.363555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca85b1-4ebf-436c-abaf-7edf471bac25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.365170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca85b1-4ebf-436c-abaf-7edf471bac25 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.375617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 786/1559] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:20.379254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcdafc-9479-4c2c-9f18-a5b736ad339d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.379254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcdafc-9479-4c2c-9f18-a5b736ad339d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.379254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcdafc-9479-4c2c-9f18-a5b736ad339d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.379564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 774/1560] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:20.379564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca85b1-4ebf-436c-abaf-7edf471bac25 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.385477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:20.393972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca85b1-4ebf-436c-abaf-7edf471bac25 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.396711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0ebcdafc-9479-4c2c-9f18-a5b736ad339d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:20.397962 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0ebcdafc-9479-4c2c-9f18-a5b736ad339d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:20.398859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430058-2b1b-4c03-8af6-4bce874f9276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.399497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430058-2b1b-4c03-8af6-4bce874f9276 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.399781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430058-2b1b-4c03-8af6-4bce874f9276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.400059 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430058-2b1b-4c03-8af6-4bce874f9276 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.400883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5430058-2b1b-4c03-8af6-4bce874f9276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.406198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5430058-2b1b-4c03-8af6-4bce874f9276 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.406329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 787/1561] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:20.407600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcdafc-9479-4c2c-9f18-a5b736ad339d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.410140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:20.418111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcdafc-9479-4c2c-9f18-a5b736ad339d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.425544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd96dc7-3648-4ec3-be36-d75e2f1f225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.426179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd96dc7-3648-4ec3-be36-d75e2f1f225c admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.426463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd96dc7-3648-4ec3-be36-d75e2f1f225c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.427126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd96dc7-3648-4ec3-be36-d75e2f1f225c admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.427450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd96dc7-3648-4ec3-be36-d75e2f1f225c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:20.427803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd96dc7-3648-4ec3-be36-d75e2f1f225c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.429419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd96dc7-3648-4ec3-be36-d75e2f1f225c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.446828 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 788/1562] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:20.447591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd96dc7-3648-4ec3-be36-d75e2f1f225c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.458146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd96dc7-3648-4ec3-be36-d75e2f1f225c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.462944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0c845-7e8e-4577-a4c4-8ecf9cc8aa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.463571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0c845-7e8e-4577-a4c4-8ecf9cc8aa3e admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.463855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0c845-7e8e-4577-a4c4-8ecf9cc8aa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.464134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0c845-7e8e-4577-a4c4-8ecf9cc8aa3e admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.464793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0c845-7e8e-4577-a4c4-8ecf9cc8aa3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.466745 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0c845-7e8e-4577-a4c4-8ecf9cc8aa3e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.519428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 775/1563] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/projects => generated 403 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:20.520527 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0679753-9a4a-4971-a3ac-8300b3712c91 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.520880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0679753-9a4a-4971-a3ac-8300b3712c91 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.521189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0679753-9a4a-4971-a3ac-8300b3712c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.540174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d0679753-9a4a-4971-a3ac-8300b3712c91 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:20.542087 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d0679753-9a4a-4971-a3ac-8300b3712c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:20.575223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 789/1564] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/users => generated 278 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:20.576055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0c845-7e8e-4577-a4c4-8ecf9cc8aa3e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.586659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0c845-7e8e-4577-a4c4-8ecf9cc8aa3e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.609212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 776/1565] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:20.610042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0679753-9a4a-4971-a3ac-8300b3712c91 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.619595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0679753-9a4a-4971-a3ac-8300b3712c91 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.625151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e6f6d-7fad-4266-a8b4-6e546711cafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.627465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e6f6d-7fad-4266-a8b4-6e546711cafd admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.627465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e6f6d-7fad-4266-a8b4-6e546711cafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.627465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e6f6d-7fad-4266-a8b4-6e546711cafd admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.628929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e6f6d-7fad-4266-a8b4-6e546711cafd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.631050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e6f6d-7fad-4266-a8b4-6e546711cafd admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.650818 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f07635-ba3e-45aa-8fd0-0f3d32186fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.651453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f07635-ba3e-45aa-8fd0-0f3d32186fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.651740 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f07635-ba3e-45aa-8fd0-0f3d32186fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.652022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f07635-ba3e-45aa-8fd0-0f3d32186fdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.652723 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f07635-ba3e-45aa-8fd0-0f3d32186fdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:20.653064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f07635-ba3e-45aa-8fd0-0f3d32186fdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.654670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f07635-ba3e-45aa-8fd0-0f3d32186fdb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.666124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 790/1566] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:20.666912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f07635-ba3e-45aa-8fd0-0f3d32186fdb admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.676468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f07635-ba3e-45aa-8fd0-0f3d32186fdb admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.681191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcff580-fd07-4e99-bc94-503fa081d6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.684758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff580-fd07-4e99-bc94-503fa081d6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.684758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff580-fd07-4e99-bc94-503fa081d6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.684758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcff580-fd07-4e99-bc94-503fa081d6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.684758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcff580-fd07-4e99-bc94-503fa081d6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.684758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcff580-fd07-4e99-bc94-503fa081d6e8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.695473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 791/1567] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:20.696245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcff580-fd07-4e99-bc94-503fa081d6e8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.706660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcff580-fd07-4e99-bc94-503fa081d6e8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.709344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 777/1568] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/projects => generated 399 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:20.710167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e6f6d-7fad-4266-a8b4-6e546711cafd admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.711494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff241b-1753-44ba-98d0-12a1fb50b417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.712136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff241b-1753-44ba-98d0-12a1fb50b417 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.712422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff241b-1753-44ba-98d0-12a1fb50b417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.712703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff241b-1753-44ba-98d0-12a1fb50b417 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.713439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff241b-1753-44ba-98d0-12a1fb50b417 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:20.713749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff241b-1753-44ba-98d0-12a1fb50b417 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.715358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff241b-1753-44ba-98d0-12a1fb50b417 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.727551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 792/1569] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:20.728579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff241b-1753-44ba-98d0-12a1fb50b417 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.740522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff241b-1753-44ba-98d0-12a1fb50b417 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.774864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e6f6d-7fad-4266-a8b4-6e546711cafd admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.781350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed362fe7-5b23-4cbf-b9fe-f998b19e12a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.781978 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed362fe7-5b23-4cbf-b9fe-f998b19e12a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.782267 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed362fe7-5b23-4cbf-b9fe-f998b19e12a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.782597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed362fe7-5b23-4cbf-b9fe-f998b19e12a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.783283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed362fe7-5b23-4cbf-b9fe-f998b19e12a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.785299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed362fe7-5b23-4cbf-b9fe-f998b19e12a1 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.800062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ea654-810e-4af3-a843-54d5cffb61db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.800694 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ea654-810e-4af3-a843-54d5cffb61db admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.801045 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ea654-810e-4af3-a843-54d5cffb61db admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.801312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ea654-810e-4af3-a843-54d5cffb61db admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.802004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ea654-810e-4af3-a843-54d5cffb61db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.803911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ea654-810e-4af3-a843-54d5cffb61db admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.895502 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 778/1570] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/projects => generated 393 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:20.896409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed362fe7-5b23-4cbf-b9fe-f998b19e12a1 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.922999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 793/1571] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/users => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:20.923847 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ea654-810e-4af3-a843-54d5cffb61db admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.934236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ea654-810e-4af3-a843-54d5cffb61db admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.948496 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:20.960328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed362fe7-5b23-4cbf-b9fe-f998b19e12a1 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.960973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:20.965175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ac884-9f96-4ce0-9441-dacc79970cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.965792 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ac884-9f96-4ce0-9441-dacc79970cec admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.966075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ac884-9f96-4ce0-9441-dacc79970cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.966345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ac884-9f96-4ce0-9441-dacc79970cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.967047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ac884-9f96-4ce0-9441-dacc79970cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.969392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ac884-9f96-4ce0-9441-dacc79970cec admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.983198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 779/1572] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:20.983198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ac884-9f96-4ce0-9441-dacc79970cec admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:20.992270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ac884-9f96-4ce0-9441-dacc79970cec admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:20.994509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ff398-0f12-420c-9551-b1c8403df8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.995170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ff398-0f12-420c-9551-b1c8403df8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.995457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ff398-0f12-420c-9551-b1c8403df8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.995896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ff398-0f12-420c-9551-b1c8403df8b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.996600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ff398-0f12-420c-9551-b1c8403df8b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:20.996909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47272a51-aff1-4444-9a8f-4a438d09e428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:20.997036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ff398-0f12-420c-9551-b1c8403df8b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:20.997536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47272a51-aff1-4444-9a8f-4a438d09e428 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:20.997826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47272a51-aff1-4444-9a8f-4a438d09e428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:20.998112 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47272a51-aff1-4444-9a8f-4a438d09e428 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:20.998637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ff398-0f12-420c-9551-b1c8403df8b8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:20.998783 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47272a51-aff1-4444-9a8f-4a438d09e428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.000753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47272a51-aff1-4444-9a8f-4a438d09e428 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.010530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 794/1573] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:21.011856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ff398-0f12-420c-9551-b1c8403df8b8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.022572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ff398-0f12-420c-9551-b1c8403df8b8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.027347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe92750-82ec-4f62-94e0-3945cd71c283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.027953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe92750-82ec-4f62-94e0-3945cd71c283 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.028238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe92750-82ec-4f62-94e0-3945cd71c283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.028517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe92750-82ec-4f62-94e0-3945cd71c283 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.029253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe92750-82ec-4f62-94e0-3945cd71c283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.031530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe92750-82ec-4f62-94e0-3945cd71c283 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.041306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 795/1574] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:21.046520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe92750-82ec-4f62-94e0-3945cd71c283 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.051680 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe92750-82ec-4f62-94e0-3945cd71c283 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.056333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69b168-2272-4eeb-a085-36eb4d54da76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.056902 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b168-2272-4eeb-a085-36eb4d54da76 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.057225 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b168-2272-4eeb-a085-36eb4d54da76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.057506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b168-2272-4eeb-a085-36eb4d54da76 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.058161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69b168-2272-4eeb-a085-36eb4d54da76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.060018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69b168-2272-4eeb-a085-36eb4d54da76 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.103091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 780/1575] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:20 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:21.104060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47272a51-aff1-4444-9a8f-4a438d09e428 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.164712 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47272a51-aff1-4444-9a8f-4a438d09e428 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.169358 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0f6ed-035f-4cf0-8918-0f3dc17eb999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.169974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0f6ed-035f-4cf0-8918-0f3dc17eb999 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.170261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0f6ed-035f-4cf0-8918-0f3dc17eb999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.170540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e0f6ed-035f-4cf0-8918-0f3dc17eb999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.171208 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0f6ed-035f-4cf0-8918-0f3dc17eb999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.173224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e0f6ed-035f-4cf0-8918-0f3dc17eb999 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.186151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 796/1576] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:21 2022] POST /identity/v3/users => generated 300 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:21.187652 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69b168-2272-4eeb-a085-36eb4d54da76 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.197436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69b168-2272-4eeb-a085-36eb4d54da76 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.202201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7428a-6d3b-4ccb-92bb-637d308348b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.203380 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7428a-6d3b-4ccb-92bb-637d308348b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.203661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7428a-6d3b-4ccb-92bb-637d308348b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.203937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7428a-6d3b-4ccb-92bb-637d308348b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.204595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7428a-6d3b-4ccb-92bb-637d308348b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.206587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7428a-6d3b-4ccb-92bb-637d308348b8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.256442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 781/1577] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:21 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:21.257443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0f6ed-035f-4cf0-8918-0f3dc17eb999 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.297021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 797/1578] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:21 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:21.297021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7428a-6d3b-4ccb-92bb-637d308348b8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.304642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7428a-6d3b-4ccb-92bb-637d308348b8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.320689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e0f6ed-035f-4cf0-8918-0f3dc17eb999 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.325926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe2137-5580-441d-b29f-d3a79f7d4a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.326639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2137-5580-441d-b29f-d3a79f7d4a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.326922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2137-5580-441d-b29f-d3a79f7d4a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.327201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2137-5580-441d-b29f-d3a79f7d4a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.328710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe2137-5580-441d-b29f-d3a79f7d4a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.330748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe2137-5580-441d-b29f-d3a79f7d4a85 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.340788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 782/1579] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:21.341601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe2137-5580-441d-b29f-d3a79f7d4a85 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.351816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe2137-5580-441d-b29f-d3a79f7d4a85 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.356863 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dca58-d502-4d3d-895f-086ec92a3887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.357603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dca58-d502-4d3d-895f-086ec92a3887 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.357894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dca58-d502-4d3d-895f-086ec92a3887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.358177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dca58-d502-4d3d-895f-086ec92a3887 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.358860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dca58-d502-4d3d-895f-086ec92a3887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.360745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dca58-d502-4d3d-895f-086ec92a3887 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.363248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1d9ea-e234-4460-b1db-e9739a9e7a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.363898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1d9ea-e234-4460-b1db-e9739a9e7a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.364177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1d9ea-e234-4460-b1db-e9739a9e7a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.364473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1d9ea-e234-4460-b1db-e9739a9e7a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.365200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1d9ea-e234-4460-b1db-e9739a9e7a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.366783 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1d9ea-e234-4460-b1db-e9739a9e7a47 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.377160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 798/1580] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:21.378025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1d9ea-e234-4460-b1db-e9739a9e7a47 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.388371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1d9ea-e234-4460-b1db-e9739a9e7a47 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.393657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ea919-1b89-467b-9473-029280631b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.394299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ea919-1b89-467b-9473-029280631b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.394608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ea919-1b89-467b-9473-029280631b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.394972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ea919-1b89-467b-9473-029280631b2d admin admin] PATH_INFO: `/v3/projects/8ef52881d75f407a942bf960230ea22e/users/d96e9a9e40c54b6a9a7f8560e4846be5/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.409354 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ea919-1b89-467b-9473-029280631b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef52881d75f407a942bf960230ea22e, user_id=d96e9a9e40c54b6a9a7f8560e4846be5, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.412153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ea919-1b89-467b-9473-029280631b2d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.438722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 783/1581] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:21 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:21.439534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dca58-d502-4d3d-895f-086ec92a3887 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.466841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:21.488364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 799/1582] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:21 2022] PUT /identity/v3/projects/8ef52881d75f407a942bf960230ea22e/users/d96e9a9e40c54b6a9a7f8560e4846be5/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:21.489382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ea919-1b89-467b-9473-029280631b2d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.502093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dca58-d502-4d3d-895f-086ec92a3887 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.537843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:21.552104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ea919-1b89-467b-9473-029280631b2d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.556886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c8e1-ed91-4cf5-96d4-a2639674ea4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.557537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c8e1-ed91-4cf5-96d4-a2639674ea4c admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.557826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c8e1-ed91-4cf5-96d4-a2639674ea4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.558206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c8e1-ed91-4cf5-96d4-a2639674ea4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.558868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3c8e1-ed91-4cf5-96d4-a2639674ea4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.560469 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3c8e1-ed91-4cf5-96d4-a2639674ea4c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.560994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f965a8a-c3e1-43b3-b04f-daf235f9f6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.561633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f965a8a-c3e1-43b3-b04f-daf235f9f6ac admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.562504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f965a8a-c3e1-43b3-b04f-daf235f9f6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.562862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f965a8a-c3e1-43b3-b04f-daf235f9f6ac admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.563554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f965a8a-c3e1-43b3-b04f-daf235f9f6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.565488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f965a8a-c3e1-43b3-b04f-daf235f9f6ac admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.571006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 800/1583] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:21.571688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c8e1-ed91-4cf5-96d4-a2639674ea4c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.582441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c8e1-ed91-4cf5-96d4-a2639674ea4c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.587162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de26d50-af2f-46c4-bcc3-8eb481f9f3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.593238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de26d50-af2f-46c4-bcc3-8eb481f9f3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.593238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de26d50-af2f-46c4-bcc3-8eb481f9f3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.593238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de26d50-af2f-46c4-bcc3-8eb481f9f3d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.593238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de26d50-af2f-46c4-bcc3-8eb481f9f3d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.593238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de26d50-af2f-46c4-bcc3-8eb481f9f3d4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.655498 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 784/1584] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:21 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:21.656341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f965a8a-c3e1-43b3-b04f-daf235f9f6ac admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.666160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f965a8a-c3e1-43b3-b04f-daf235f9f6ac admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.670928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb60a3b-c828-4ff5-b799-642cc0335ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.672018 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb60a3b-c828-4ff5-b799-642cc0335ebd admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.672304 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb60a3b-c828-4ff5-b799-642cc0335ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.672583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb60a3b-c828-4ff5-b799-642cc0335ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.673358 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb60a3b-c828-4ff5-b799-642cc0335ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.674909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb60a3b-c828-4ff5-b799-642cc0335ebd admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.680013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 801/1585] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:21 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:21.681391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de26d50-af2f-46c4-bcc3-8eb481f9f3d4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.688627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 785/1586] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:21.691845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de26d50-af2f-46c4-bcc3-8eb481f9f3d4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.692837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb60a3b-c828-4ff5-b799-642cc0335ebd admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.696691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3e787-7de4-4d19-861b-cde462d50870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.697333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3e787-7de4-4d19-861b-cde462d50870 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.697623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3e787-7de4-4d19-861b-cde462d50870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.698136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3e787-7de4-4d19-861b-cde462d50870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.699321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3e787-7de4-4d19-861b-cde462d50870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.700554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3e787-7de4-4d19-861b-cde462d50870 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.702578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb60a3b-c828-4ff5-b799-642cc0335ebd admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.707563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e5cd8-af2d-49fd-bc90-05b8189501b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.708226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e5cd8-af2d-49fd-bc90-05b8189501b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.708522 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e5cd8-af2d-49fd-bc90-05b8189501b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.708804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e5cd8-af2d-49fd-bc90-05b8189501b1 admin admin] PATH_INFO: `/v3/projects/6b4da7db7b464b4f9f3e11b77b934db3/users/9a8e746ee67b45bd8d0988de9287407f/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.712176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 802/1587] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:21.712903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3e787-7de4-4d19-861b-cde462d50870 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.722969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3e787-7de4-4d19-861b-cde462d50870 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.725091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e5cd8-af2d-49fd-bc90-05b8189501b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b4da7db7b464b4f9f3e11b77b934db3, user_id=9a8e746ee67b45bd8d0988de9287407f, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.726161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-707e5cd8-af2d-49fd-bc90-05b8189501b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:12:21.726161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: warnings.warn(msg) Jul 28 18:12:21.726161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: Jul 28 18:12:21.727648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db13d46-7b0a-450b-bc47-df91b4a2d18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.728123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e5cd8-af2d-49fd-bc90-05b8189501b1 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.728250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13d46-7b0a-450b-bc47-df91b4a2d18e admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.728501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13d46-7b0a-450b-bc47-df91b4a2d18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.728778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db13d46-7b0a-450b-bc47-df91b4a2d18e admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.729522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db13d46-7b0a-450b-bc47-df91b4a2d18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.731393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db13d46-7b0a-450b-bc47-df91b4a2d18e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.803112 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 786/1588] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:21 2022] PUT /identity/v3/projects/6b4da7db7b464b4f9f3e11b77b934db3/users/9a8e746ee67b45bd8d0988de9287407f/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:21.804555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd880c7-a8b6-4d5d-9a0b-5726465817d3 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.804852 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd880c7-a8b6-4d5d-9a0b-5726465817d3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.805241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd880c7-a8b6-4d5d-9a0b-5726465817d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.824516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6fd880c7-a8b6-4d5d-9a0b-5726465817d3 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:21.825764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6fd880c7-a8b6-4d5d-9a0b-5726465817d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:21.838347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 803/1589] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:21 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:21.839197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db13d46-7b0a-450b-bc47-df91b4a2d18e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.849296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db13d46-7b0a-450b-bc47-df91b4a2d18e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.887094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 787/1590] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:21.888305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd880c7-a8b6-4d5d-9a0b-5726465817d3 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.898037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd880c7-a8b6-4d5d-9a0b-5726465817d3 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.902882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b60b5-bf61-4b54-91a1-05d061997062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.903501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b60b5-bf61-4b54-91a1-05d061997062 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.903788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b60b5-bf61-4b54-91a1-05d061997062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.904071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b60b5-bf61-4b54-91a1-05d061997062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.904775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b60b5-bf61-4b54-91a1-05d061997062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.906914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b60b5-bf61-4b54-91a1-05d061997062 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.909919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424c710-0cd9-4822-9a46-0e91fea13785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.910581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424c710-0cd9-4822-9a46-0e91fea13785 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.910868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424c710-0cd9-4822-9a46-0e91fea13785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.911148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424c710-0cd9-4822-9a46-0e91fea13785 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.911854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424c710-0cd9-4822-9a46-0e91fea13785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.913502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424c710-0cd9-4822-9a46-0e91fea13785 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.917010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 788/1591] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:21.917679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b60b5-bf61-4b54-91a1-05d061997062 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.928472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 804/1592] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:21.928472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424c710-0cd9-4822-9a46-0e91fea13785 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.931109 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b60b5-bf61-4b54-91a1-05d061997062 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.935915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3924e-c23e-4f91-9edc-fcd19659a4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.936158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424c710-0cd9-4822-9a46-0e91fea13785 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.936519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3924e-c23e-4f91-9edc-fcd19659a4fd admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.936806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3924e-c23e-4f91-9edc-fcd19659a4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.937137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3924e-c23e-4f91-9edc-fcd19659a4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.937835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3924e-c23e-4f91-9edc-fcd19659a4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.939422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3924e-c23e-4f91-9edc-fcd19659a4fd admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.941036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593e4b4-cc2b-47e8-a0a8-1e7651936846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.941702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593e4b4-cc2b-47e8-a0a8-1e7651936846 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.942056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593e4b4-cc2b-47e8-a0a8-1e7651936846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.942407 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593e4b4-cc2b-47e8-a0a8-1e7651936846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.943196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593e4b4-cc2b-47e8-a0a8-1e7651936846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.944858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593e4b4-cc2b-47e8-a0a8-1e7651936846 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.949394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 789/1593] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:21.950606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3924e-c23e-4f91-9edc-fcd19659a4fd admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:21.956183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:21.956315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 805/1594] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:21.960103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3924e-c23e-4f91-9edc-fcd19659a4fd admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:21.960895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50526-ec7b-4c01-bc79-13f97c00abcf None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.961216 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50526-ec7b-4c01-bc79-13f97c00abcf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.961500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50526-ec7b-4c01-bc79-13f97c00abcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.964834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834ec92-4690-47bd-861b-f335391ee714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:21.965520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834ec92-4690-47bd-861b-f335391ee714 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:21.971579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834ec92-4690-47bd-861b-f335391ee714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:21.971579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834ec92-4690-47bd-861b-f335391ee714 admin admin] PATH_INFO: `/v3/projects/c498ba33da3e47d68c406d75b01bf83a/users/7c22ee0181454cb9bd17e7fcbae27191/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:21.982281 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834ec92-4690-47bd-861b-f335391ee714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c498ba33da3e47d68c406d75b01bf83a, user_id=7c22ee0181454cb9bd17e7fcbae27191, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:21.985614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834ec92-4690-47bd-861b-f335391ee714 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:21.990234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2da50526-ec7b-4c01-bc79-13f97c00abcf None None] MFA Rules not processed for user `9a8e746ee67b45bd8d0988de9287407f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:21.991459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2da50526-ec7b-4c01-bc79-13f97c00abcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:22.054170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 790/1595] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:21 2022] PUT /identity/v3/projects/c498ba33da3e47d68c406d75b01bf83a/users/7c22ee0181454cb9bd17e7fcbae27191/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:22.058399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 806/1596] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:21 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:22.058543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834ec92-4690-47bd-861b-f335391ee714 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.058657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da50526-ec7b-4c01-bc79-13f97c00abcf None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.058657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2da50526-ec7b-4c01-bc79-13f97c00abcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:22.075761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:22.119758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da50526-ec7b-4c01-bc79-13f97c00abcf None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.125662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834ec92-4690-47bd-861b-f335391ee714 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.130519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a31f80-7877-438a-a8ed-8995ce5a7217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.131359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31f80-7877-438a-a8ed-8995ce5a7217 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.131625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31f80-7877-438a-a8ed-8995ce5a7217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.131901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31f80-7877-438a-a8ed-8995ce5a7217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.132731 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a31f80-7877-438a-a8ed-8995ce5a7217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.134422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a31f80-7877-438a-a8ed-8995ce5a7217 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.144716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 791/1597] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:22.145577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a31f80-7877-438a-a8ed-8995ce5a7217 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.155379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a31f80-7877-438a-a8ed-8995ce5a7217 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.160713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f0a66-7b1c-4291-bd26-702310c61ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.161539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f0a66-7b1c-4291-bd26-702310c61ace admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.161828 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f0a66-7b1c-4291-bd26-702310c61ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.162107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f0a66-7b1c-4291-bd26-702310c61ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.162801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f0a66-7b1c-4291-bd26-702310c61ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.164385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f0a66-7b1c-4291-bd26-702310c61ace admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.170962 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c93bc1-9a77-4a9b-892f-ebbac4b10f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.172126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c93bc1-9a77-4a9b-892f-ebbac4b10f40 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.172407 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c93bc1-9a77-4a9b-892f-ebbac4b10f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.172679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c93bc1-9a77-4a9b-892f-ebbac4b10f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.175192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 792/1598] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:22.175757 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f0a66-7b1c-4291-bd26-702310c61ace admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.185023 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c93bc1-9a77-4a9b-892f-ebbac4b10f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.186391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c93bc1-9a77-4a9b-892f-ebbac4b10f40 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.188205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f0a66-7b1c-4291-bd26-702310c61ace admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.192856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111806a-9895-4363-a7ce-74a584fa1ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.193530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111806a-9895-4363-a7ce-74a584fa1ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.193817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111806a-9895-4363-a7ce-74a584fa1ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.194096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111806a-9895-4363-a7ce-74a584fa1ba5 admin admin] PATH_INFO: `/v3/projects/128b148a8b70445d907e6f8433411929/users/664cbcf779a84c3199b38770f3aa5848/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.195929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 807/1599] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:22.196713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c93bc1-9a77-4a9b-892f-ebbac4b10f40 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.205730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c93bc1-9a77-4a9b-892f-ebbac4b10f40 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.209238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111806a-9895-4363-a7ce-74a584fa1ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=128b148a8b70445d907e6f8433411929, user_id=664cbcf779a84c3199b38770f3aa5848, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.210595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f6d40-e4dc-49aa-ba03-c8a52987757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.211203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f6d40-e4dc-49aa-ba03-c8a52987757f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.211486 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f6d40-e4dc-49aa-ba03-c8a52987757f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.211765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f6d40-e4dc-49aa-ba03-c8a52987757f admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.211995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111806a-9895-4363-a7ce-74a584fa1ba5 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.212459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f6d40-e4dc-49aa-ba03-c8a52987757f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.214167 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f6d40-e4dc-49aa-ba03-c8a52987757f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.225551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 808/1600] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:22.226566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f6d40-e4dc-49aa-ba03-c8a52987757f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.236898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f6d40-e4dc-49aa-ba03-c8a52987757f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.241473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51aac1-3c69-4c44-93c1-52a93f33aaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.242104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51aac1-3c69-4c44-93c1-52a93f33aaa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.242386 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51aac1-3c69-4c44-93c1-52a93f33aaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.242664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51aac1-3c69-4c44-93c1-52a93f33aaa0 admin admin] PATH_INFO: `/v3/projects/b52f26ab8b934756b0c74d681d08d04f/users/5ef698c4f57f44d387550921a288d7ab/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.257200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51aac1-3c69-4c44-93c1-52a93f33aaa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52f26ab8b934756b0c74d681d08d04f, user_id=5ef698c4f57f44d387550921a288d7ab, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.260005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51aac1-3c69-4c44-93c1-52a93f33aaa0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.277539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 793/1601] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:22 2022] PUT /identity/v3/projects/128b148a8b70445d907e6f8433411929/users/664cbcf779a84c3199b38770f3aa5848/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:22.278504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111806a-9895-4363-a7ce-74a584fa1ba5 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.342871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111806a-9895-4363-a7ce-74a584fa1ba5 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.348470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a72f9-4a02-4779-b602-399d8c5505cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.349127 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a72f9-4a02-4779-b602-399d8c5505cd admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.349421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a72f9-4a02-4779-b602-399d8c5505cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.349699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a72f9-4a02-4779-b602-399d8c5505cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.350439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a72f9-4a02-4779-b602-399d8c5505cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.352353 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a72f9-4a02-4779-b602-399d8c5505cd admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.357968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 809/1602] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:22 2022] PUT /identity/v3/projects/b52f26ab8b934756b0c74d681d08d04f/users/5ef698c4f57f44d387550921a288d7ab/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:22.362010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1564e3c-3e56-418e-b9dd-3ac590362182 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.362010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1564e3c-3e56-418e-b9dd-3ac590362182 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.362010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1564e3c-3e56-418e-b9dd-3ac590362182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.388788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c1564e3c-3e56-418e-b9dd-3ac590362182 None None] MFA Rules not processed for user `7c22ee0181454cb9bd17e7fcbae27191`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:22.394448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c1564e3c-3e56-418e-b9dd-3ac590362182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:22.449292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 794/1603] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:22 2022] POST /identity/v3/projects => generated 357 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:22.450066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a72f9-4a02-4779-b602-399d8c5505cd admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.457593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 810/1604] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:22.458553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1564e3c-3e56-418e-b9dd-3ac590362182 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.485996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:22.498091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:22.524947 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a72f9-4a02-4779-b602-399d8c5505cd admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.530186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a078a5d-4537-4449-82d0-cb66e45bdab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.531680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078a5d-4537-4449-82d0-cb66e45bdab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.532025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078a5d-4537-4449-82d0-cb66e45bdab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.538195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078a5d-4537-4449-82d0-cb66e45bdab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.538195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a078a5d-4537-4449-82d0-cb66e45bdab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.538195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a078a5d-4537-4449-82d0-cb66e45bdab6 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.538593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1564e3c-3e56-418e-b9dd-3ac590362182 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.542151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190395f-3eca-426e-8eb6-4e84759f642f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.542905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190395f-3eca-426e-8eb6-4e84759f642f admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.543187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190395f-3eca-426e-8eb6-4e84759f642f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.543463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190395f-3eca-426e-8eb6-4e84759f642f admin admin] PATH_INFO: `/v3/projects/9e962e97777048b79415ba13adcc95be/users/ca7d773290a9483d91ebefad341addee/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.547001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 795/1605] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:22.547808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a078a5d-4537-4449-82d0-cb66e45bdab6 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.561865 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190395f-3eca-426e-8eb6-4e84759f642f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e962e97777048b79415ba13adcc95be, user_id=ca7d773290a9483d91ebefad341addee, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.561968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a078a5d-4537-4449-82d0-cb66e45bdab6 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.567404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190395f-3eca-426e-8eb6-4e84759f642f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.567576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50305ae2-b5c3-407c-bb0f-3d7e6288b750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.567576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50305ae2-b5c3-407c-bb0f-3d7e6288b750 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.567576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50305ae2-b5c3-407c-bb0f-3d7e6288b750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.567576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50305ae2-b5c3-407c-bb0f-3d7e6288b750 admin admin] PATH_INFO: `/v3/projects/27b91f25d2be408ba9e8d667a574a7a9/users/b9874df18af245688db1137b6f1b0bff/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.585820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50305ae2-b5c3-407c-bb0f-3d7e6288b750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b91f25d2be408ba9e8d667a574a7a9, user_id=b9874df18af245688db1137b6f1b0bff, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.588527 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50305ae2-b5c3-407c-bb0f-3d7e6288b750 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.643806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 811/1606] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:22 2022] PUT /identity/v3/projects/9e962e97777048b79415ba13adcc95be/users/ca7d773290a9483d91ebefad341addee/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:22.645159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220b712-6563-4fe4-81e3-026430161f96 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.645510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220b712-6563-4fe4-81e3-026430161f96 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.645842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220b712-6563-4fe4-81e3-026430161f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.670902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 796/1607] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:22 2022] PUT /identity/v3/projects/27b91f25d2be408ba9e8d667a574a7a9/users/b9874df18af245688db1137b6f1b0bff/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:22.673747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6be02-ef37-44ed-b1d1-302f87249724 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.673747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6be02-ef37-44ed-b1d1-302f87249724 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.674040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6be02-ef37-44ed-b1d1-302f87249724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.674197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6220b712-6563-4fe4-81e3-026430161f96 None None] MFA Rules not processed for user `664cbcf779a84c3199b38770f3aa5848`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:22.675336 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6220b712-6563-4fe4-81e3-026430161f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:22.692397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-25b6be02-ef37-44ed-b1d1-302f87249724 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:22.693639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-25b6be02-ef37-44ed-b1d1-302f87249724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:22.742785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 812/1608] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:22.742785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220b712-6563-4fe4-81e3-026430161f96 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.765412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 797/1609] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:22.765412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6be02-ef37-44ed-b1d1-302f87249724 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.778960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6be02-ef37-44ed-b1d1-302f87249724 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.784656 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd96d29-94b1-4a5e-812c-025ce7d4b470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.785336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd96d29-94b1-4a5e-812c-025ce7d4b470 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.785688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd96d29-94b1-4a5e-812c-025ce7d4b470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.786033 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd96d29-94b1-4a5e-812c-025ce7d4b470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.786877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd96d29-94b1-4a5e-812c-025ce7d4b470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.788903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd96d29-94b1-4a5e-812c-025ce7d4b470 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.822556 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220b712-6563-4fe4-81e3-026430161f96 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.827373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482da6e-b1f4-4e6a-8660-76cb3cf2f1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.828620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482da6e-b1f4-4e6a-8660-76cb3cf2f1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.828620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482da6e-b1f4-4e6a-8660-76cb3cf2f1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.828867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482da6e-b1f4-4e6a-8660-76cb3cf2f1f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.829497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482da6e-b1f4-4e6a-8660-76cb3cf2f1f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.831464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482da6e-b1f4-4e6a-8660-76cb3cf2f1f4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:22.886921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 798/1610] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:22 2022] POST /identity/v3/projects => generated 403 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:22.887839 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd96d29-94b1-4a5e-812c-025ce7d4b470 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:22.942256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 813/1611] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:22 2022] POST /identity/v3/users => generated 285 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:22.943533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240ae9f-6d05-467e-8f81-b148f79409c5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.943826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240ae9f-6d05-467e-8f81-b148f79409c5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.944261 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240ae9f-6d05-467e-8f81-b148f79409c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.951824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd96d29-94b1-4a5e-812c-025ce7d4b470 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:22.956801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ae148-7ceb-4946-9cd1-a48a0194250b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:22.957499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ae148-7ceb-4946-9cd1-a48a0194250b admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.957787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ae148-7ceb-4946-9cd1-a48a0194250b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.958066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ae148-7ceb-4946-9cd1-a48a0194250b admin admin] PATH_INFO: `/v3/projects/79a097f0698e4b0fa113b651a01051b7/users/a4e1efaae5184880a180d4eda894426e/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.964240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4240ae9f-6d05-467e-8f81-b148f79409c5 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:22.965630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4240ae9f-6d05-467e-8f81-b148f79409c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:22.978386 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 814/1612] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:22.978604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ae148-7ceb-4946-9cd1-a48a0194250b admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a097f0698e4b0fa113b651a01051b7, user_id=a4e1efaae5184880a180d4eda894426e, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:22.978711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6255d-32f4-4ac7-897e-41e4833817c6 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:22.978711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6255d-32f4-4ac7-897e-41e4833817c6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:22.978711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6255d-32f4-4ac7-897e-41e4833817c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:22.978966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ae148-7ceb-4946-9cd1-a48a0194250b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.001667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6fc6255d-32f4-4ac7-897e-41e4833817c6 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:23.002896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6fc6255d-32f4-4ac7-897e-41e4833817c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:23.010814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 815/1613] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:23.011728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6255d-32f4-4ac7-897e-41e4833817c6 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.022436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6255d-32f4-4ac7-897e-41e4833817c6 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:23.027139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dabea-1474-4693-b176-fdd0ce4d72cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:23.027755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dabea-1474-4693-b176-fdd0ce4d72cb admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.028036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dabea-1474-4693-b176-fdd0ce4d72cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.028311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dabea-1474-4693-b176-fdd0ce4d72cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.029034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dabea-1474-4693-b176-fdd0ce4d72cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:23.031673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dabea-1474-4693-b176-fdd0ce4d72cb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.043680 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:23.046289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:23.052474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 799/1614] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:22 2022] PUT /identity/v3/projects/79a097f0698e4b0fa113b651a01051b7/users/a4e1efaae5184880a180d4eda894426e/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:23.053896 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ae148-7ceb-4946-9cd1-a48a0194250b admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.119933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ae148-7ceb-4946-9cd1-a48a0194250b admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:23.123653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac8c67-dadc-419b-8382-e70d1f96c5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:23.124299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac8c67-dadc-419b-8382-e70d1f96c5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.124587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac8c67-dadc-419b-8382-e70d1f96c5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.124864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac8c67-dadc-419b-8382-e70d1f96c5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.158762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 816/1615] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:23 2022] POST /identity/v3/projects => generated 401 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:23.159607 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dabea-1474-4693-b176-fdd0ce4d72cb admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.188713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac8c67-dadc-419b-8382-e70d1f96c5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:23.190206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac8c67-dadc-419b-8382-e70d1f96c5f6 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.199831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 800/1616] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:23.200632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac8c67-dadc-419b-8382-e70d1f96c5f6 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.210416 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac8c67-dadc-419b-8382-e70d1f96c5f6 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:23.227443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dabea-1474-4693-b176-fdd0ce4d72cb admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:23.232557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9106d15-1568-4724-9274-7624129bdf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:23.233329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9106d15-1568-4724-9274-7624129bdf14 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.233627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9106d15-1568-4724-9274-7624129bdf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.233942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9106d15-1568-4724-9274-7624129bdf14 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.234618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9106d15-1568-4724-9274-7624129bdf14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:23.236599 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9106d15-1568-4724-9274-7624129bdf14 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.273917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f40d7-51c9-4a85-bd40-2d89076e27ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:23.274635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f40d7-51c9-4a85-bd40-2d89076e27ff admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.274925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f40d7-51c9-4a85-bd40-2d89076e27ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.275205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f40d7-51c9-4a85-bd40-2d89076e27ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.275913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f40d7-51c9-4a85-bd40-2d89076e27ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:23.277564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f40d7-51c9-4a85-bd40-2d89076e27ff admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.288390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 801/1617] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:23.289073 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f40d7-51c9-4a85-bd40-2d89076e27ff admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.341617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 817/1618] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:23 2022] POST /identity/v3/users => generated 308 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:23.342844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9106d15-1568-4724-9274-7624129bdf14 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.351512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:23.358114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f40d7-51c9-4a85-bd40-2d89076e27ff admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:23.365548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76741ea5-f76b-462c-b4d7-1d4d83928cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:23.365548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76741ea5-f76b-462c-b4d7-1d4d83928cbd service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.365548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76741ea5-f76b-462c-b4d7-1d4d83928cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.365548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76741ea5-f76b-462c-b4d7-1d4d83928cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.373313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76741ea5-f76b-462c-b4d7-1d4d83928cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:23.375468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76741ea5-f76b-462c-b4d7-1d4d83928cbd service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.386436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 802/1619] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:23.387804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee3019-5815-4f6c-8fdb-1a399836bccd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.388058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee3019-5815-4f6c-8fdb-1a399836bccd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.388336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee3019-5815-4f6c-8fdb-1a399836bccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.428874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9106d15-1568-4724-9274-7624129bdf14 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:23.431900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f5ee3019-5815-4f6c-8fdb-1a399836bccd None None] MFA Rules not processed for user `a4e1efaae5184880a180d4eda894426e`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:23.433379 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad7759-3973-4c2f-a011-9d990eb04831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:23.434013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad7759-3973-4c2f-a011-9d990eb04831 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.434302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f5ee3019-5815-4f6c-8fdb-1a399836bccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:23.434513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad7759-3973-4c2f-a011-9d990eb04831 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.434980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad7759-3973-4c2f-a011-9d990eb04831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.450284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad7759-3973-4c2f-a011-9d990eb04831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:23.451748 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad7759-3973-4c2f-a011-9d990eb04831 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.462308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 818/1620] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:23.462914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad7759-3973-4c2f-a011-9d990eb04831 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.472501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad7759-3973-4c2f-a011-9d990eb04831 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:23.477550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bd2fa-65d8-4cf1-bf47-f2e547d4dfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:23.478162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bd2fa-65d8-4cf1-bf47-f2e547d4dfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.478445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bd2fa-65d8-4cf1-bf47-f2e547d4dfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.478717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bd2fa-65d8-4cf1-bf47-f2e547d4dfe5 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.479528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bd2fa-65d8-4cf1-bf47-f2e547d4dfe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:23.481464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bd2fa-65d8-4cf1-bf47-f2e547d4dfe5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.512191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 803/1621] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:23.513364 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee3019-5815-4f6c-8fdb-1a399836bccd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.531779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee3019-5815-4f6c-8fdb-1a399836bccd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:23.538653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f507b67-335e-46bd-bd4d-a03d88413175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:23.540077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f507b67-335e-46bd-bd4d-a03d88413175 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.540440 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f507b67-335e-46bd-bd4d-a03d88413175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.540790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f507b67-335e-46bd-bd4d-a03d88413175 admin admin] PATH_INFO: `/v3/projects/1d3f486da27d4806a0e016f8f265a486/users/e9cf8119e3084450bd33f117fc2d9089/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.563884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f507b67-335e-46bd-bd4d-a03d88413175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3f486da27d4806a0e016f8f265a486, user_id=e9cf8119e3084450bd33f117fc2d9089, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:23.566878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f507b67-335e-46bd-bd4d-a03d88413175 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.735626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 819/1622] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:23 2022] POST /identity/v3/users => generated 307 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:23.738530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bd2fa-65d8-4cf1-bf47-f2e547d4dfe5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.747529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bd2fa-65d8-4cf1-bf47-f2e547d4dfe5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:23.752840 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd81a7-c212-48a8-a339-6f5c8228ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:23.753661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd81a7-c212-48a8-a339-6f5c8228ff8f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.753827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd81a7-c212-48a8-a339-6f5c8228ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.754206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd81a7-c212-48a8-a339-6f5c8228ff8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.755021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd81a7-c212-48a8-a339-6f5c8228ff8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:23.764150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 804/1623] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:23 2022] PUT /identity/v3/projects/1d3f486da27d4806a0e016f8f265a486/users/e9cf8119e3084450bd33f117fc2d9089/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:23.767145 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92cabd-33c9-4195-9a3f-0fa1f3e450e4 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.767145 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92cabd-33c9-4195-9a3f-0fa1f3e450e4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.767145 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92cabd-33c9-4195-9a3f-0fa1f3e450e4 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.767423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd81a7-c212-48a8-a339-6f5c8228ff8f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.768633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 805/1624] 10.222.0.86 () {54 vars in 962 bytes} [Thu Jul 28 18:12:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:12:23.773146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92cabd-33c9-4195-9a3f-0fa1f3e450e4 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.800959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 820/1625] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:23.800959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd81a7-c212-48a8-a339-6f5c8228ff8f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.810270 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd81a7-c212-48a8-a339-6f5c8228ff8f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:23.893416 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:23.906464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c19cc-1158-4cad-a5e5-0462c0485a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:23.908152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c19cc-1158-4cad-a5e5-0462c0485a7a admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.908308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c19cc-1158-4cad-a5e5-0462c0485a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.908487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c19cc-1158-4cad-a5e5-0462c0485a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.910511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c19cc-1158-4cad-a5e5-0462c0485a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:23.912145 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92cabd-33c9-4195-9a3f-0fa1f3e450e4 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:23.913905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c19cc-1158-4cad-a5e5-0462c0485a7a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.924149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd23fcc-3ee5-4526-bbec-995aeed2f284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:23.924479 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd23fcc-3ee5-4526-bbec-995aeed2f284 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.924762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd23fcc-3ee5-4526-bbec-995aeed2f284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.925061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd23fcc-3ee5-4526-bbec-995aeed2f284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.925778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd23fcc-3ee5-4526-bbec-995aeed2f284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:23.940743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd23fcc-3ee5-4526-bbec-995aeed2f284 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:23.961914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 821/1626] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:23.961914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb424f99-6929-4597-b5be-0ed71e99ad9e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:23.961914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb424f99-6929-4597-b5be-0ed71e99ad9e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:23.961914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb424f99-6929-4597-b5be-0ed71e99ad9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:23.977071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 806/1627] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:23.984718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd23fcc-3ee5-4526-bbec-995aeed2f284 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:23.997435 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd23fcc-3ee5-4526-bbec-995aeed2f284 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:24.004303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee16ab-13ad-4845-8b9f-077eab7e0125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:24.005070 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee16ab-13ad-4845-8b9f-077eab7e0125 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:24.005461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee16ab-13ad-4845-8b9f-077eab7e0125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:24.005860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee16ab-13ad-4845-8b9f-077eab7e0125 admin admin] PATH_INFO: `/v3/projects/cb023f437dcc419c9be98f6036a14609/users/02ee0257f9a44b35becc1422e2c5975a/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:24.037769 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee16ab-13ad-4845-8b9f-077eab7e0125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb023f437dcc419c9be98f6036a14609, user_id=02ee0257f9a44b35becc1422e2c5975a, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:24.043573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee16ab-13ad-4845-8b9f-077eab7e0125 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:24.073381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-cb424f99-6929-4597-b5be-0ed71e99ad9e None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:24.074744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cb424f99-6929-4597-b5be-0ed71e99ad9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:24.149257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 807/1628] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:23 2022] PUT /identity/v3/projects/cb023f437dcc419c9be98f6036a14609/users/02ee0257f9a44b35becc1422e2c5975a/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:24.149892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee16ab-13ad-4845-8b9f-077eab7e0125 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:24.164037 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 822/1629] 10.222.0.86 () {58 vars in 1059 bytes} [Thu Jul 28 18:12:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:24.165993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb424f99-6929-4597-b5be-0ed71e99ad9e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:24.178642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb424f99-6929-4597-b5be-0ed71e99ad9e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:24.230567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:24.262142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bda7df-0320-4a83-8fc2-654c3ecd5202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:24.266056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bda7df-0320-4a83-8fc2-654c3ecd5202 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:24.266880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bda7df-0320-4a83-8fc2-654c3ecd5202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:24.267163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bda7df-0320-4a83-8fc2-654c3ecd5202 admin admin] PATH_INFO: `/v3/projects/e31226e571684364a55b376c28f089d2/users/30dbf3429b2143b8882435cd0ff718e5/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:24.287437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee16ab-13ad-4845-8b9f-077eab7e0125 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:24.294540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bda7df-0320-4a83-8fc2-654c3ecd5202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e31226e571684364a55b376c28f089d2, user_id=30dbf3429b2143b8882435cd0ff718e5, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:24.298060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bda7df-0320-4a83-8fc2-654c3ecd5202 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:24.299777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aaa042-1685-4296-a3a8-604b559a74b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:24.300477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aaa042-1685-4296-a3a8-604b559a74b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:24.303383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aaa042-1685-4296-a3a8-604b559a74b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:24.303695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aaa042-1685-4296-a3a8-604b559a74b5 admin admin] PATH_INFO: `/v3/projects/1d3f486da27d4806a0e016f8f265a486/users/e9cf8119e3084450bd33f117fc2d9089/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:24.317521 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aaa042-1685-4296-a3a8-604b559a74b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3f486da27d4806a0e016f8f265a486, user_id=e9cf8119e3084450bd33f117fc2d9089, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:24.337680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aaa042-1685-4296-a3a8-604b559a74b5 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:24.369277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 808/1630] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:24 2022] PUT /identity/v3/projects/1d3f486da27d4806a0e016f8f265a486/users/e9cf8119e3084450bd33f117fc2d9089/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:24.370362 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aaa042-1685-4296-a3a8-604b559a74b5 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:24.415022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 823/1631] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:24 2022] PUT /identity/v3/projects/e31226e571684364a55b376c28f089d2/users/30dbf3429b2143b8882435cd0ff718e5/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:24.416011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bda7df-0320-4a83-8fc2-654c3ecd5202 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:24.493307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aaa042-1685-4296-a3a8-604b559a74b5 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:24.572618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf163e6-0d79-4271-9ac8-f8a609ed0224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:24.573407 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf163e6-0d79-4271-9ac8-f8a609ed0224 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:24.573690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf163e6-0d79-4271-9ac8-f8a609ed0224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:24.573975 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf163e6-0d79-4271-9ac8-f8a609ed0224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:24.574916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf163e6-0d79-4271-9ac8-f8a609ed0224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:24.578205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf163e6-0d79-4271-9ac8-f8a609ed0224 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:24.585003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bda7df-0320-4a83-8fc2-654c3ecd5202 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:24.597174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 809/1632] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:24.597174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd01be-9464-4a3c-afc4-0f0c3fc43252 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:24.597174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd01be-9464-4a3c-afc4-0f0c3fc43252 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:24.603827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd01be-9464-4a3c-afc4-0f0c3fc43252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:24.603929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3293e5-c851-4900-92ec-f2851ff6a3f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:24.603929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3293e5-c851-4900-92ec-f2851ff6a3f5 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:24.603929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3293e5-c851-4900-92ec-f2851ff6a3f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:24.603929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3293e5-c851-4900-92ec-f2851ff6a3f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:24.661186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-fafd01be-9464-4a3c-afc4-0f0c3fc43252 None None] MFA Rules not processed for user `e9cf8119e3084450bd33f117fc2d9089`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:24.665678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fafd01be-9464-4a3c-afc4-0f0c3fc43252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:24.703121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3293e5-c851-4900-92ec-f2851ff6a3f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:24.708231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3293e5-c851-4900-92ec-f2851ff6a3f5 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:24.743565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 824/1633] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:24.745625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3293e5-c851-4900-92ec-f2851ff6a3f5 service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:24.763224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:24.790683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3293e5-c851-4900-92ec-f2851ff6a3f5 service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:24.807877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54feeb-be75-4ebc-a716-76b010ae391c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:24.808600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54feeb-be75-4ebc-a716-76b010ae391c admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:24.808891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54feeb-be75-4ebc-a716-76b010ae391c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:24.811007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54feeb-be75-4ebc-a716-76b010ae391c admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:24.813283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54feeb-be75-4ebc-a716-76b010ae391c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:24.816139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54feeb-be75-4ebc-a716-76b010ae391c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:24.846353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 825/1634] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:24.846353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54feeb-be75-4ebc-a716-76b010ae391c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:24.860723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 810/1635] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:24 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:24.860723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd01be-9464-4a3c-afc4-0f0c3fc43252 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:24.883741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54feeb-be75-4ebc-a716-76b010ae391c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:24.886057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd01be-9464-4a3c-afc4-0f0c3fc43252 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:24.901490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e5055-3b37-4632-9abe-2ed257ced83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:24.909293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e5055-3b37-4632-9abe-2ed257ced83d admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:24.909293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e5055-3b37-4632-9abe-2ed257ced83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:24.909293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e5055-3b37-4632-9abe-2ed257ced83d admin admin] PATH_INFO: `/v3/projects/cb023f437dcc419c9be98f6036a14609/users/02ee0257f9a44b35becc1422e2c5975a/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:24.909591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08382b38-73b6-4658-bf69-4f89cc322d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:24.909591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08382b38-73b6-4658-bf69-4f89cc322d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:24.910491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e5055-3b37-4632-9abe-2ed257ced83d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb023f437dcc419c9be98f6036a14609, user_id=02ee0257f9a44b35becc1422e2c5975a, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:24.912349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08382b38-73b6-4658-bf69-4f89cc322d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:24.912646 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08382b38-73b6-4658-bf69-4f89cc322d6d admin admin] PATH_INFO: `/v3/projects/e31226e571684364a55b376c28f089d2/users/30dbf3429b2143b8882435cd0ff718e5/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:24.916673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08382b38-73b6-4658-bf69-4f89cc322d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e31226e571684364a55b376c28f089d2, user_id=30dbf3429b2143b8882435cd0ff718e5, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:24.918992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e5055-3b37-4632-9abe-2ed257ced83d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:24.925192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08382b38-73b6-4658-bf69-4f89cc322d6d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:24.945824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 826/1636] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:24 2022] PUT /identity/v3/projects/cb023f437dcc419c9be98f6036a14609/users/02ee0257f9a44b35becc1422e2c5975a/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:24.948245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e5055-3b37-4632-9abe-2ed257ced83d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:24.952835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 811/1637] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:24 2022] PUT /identity/v3/projects/e31226e571684364a55b376c28f089d2/users/30dbf3429b2143b8882435cd0ff718e5/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:24.955441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4db58e-c602-450a-92d1-c151a264db41 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:24.955846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4db58e-c602-450a-92d1-c151a264db41 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:24.956344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4db58e-c602-450a-92d1-c151a264db41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:25.039718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-af4db58e-c602-450a-92d1-c151a264db41 None None] MFA Rules not processed for user `02ee0257f9a44b35becc1422e2c5975a`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:25.043354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-af4db58e-c602-450a-92d1-c151a264db41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:25.106247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e5055-3b37-4632-9abe-2ed257ced83d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:25.264256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 812/1638] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:24 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:25.264256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae44a-c065-4126-a0f2-ab12cdf4db0a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:25.264256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae44a-c065-4126-a0f2-ab12cdf4db0a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:25.264256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae44a-c065-4126-a0f2-ab12cdf4db0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:25.264256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c1cae44a-c065-4126-a0f2-ab12cdf4db0a None None] MFA Rules not processed for user `30dbf3429b2143b8882435cd0ff718e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:25.264256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c1cae44a-c065-4126-a0f2-ab12cdf4db0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:25.266059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2a150-2f63-4fbc-9a81-59a87328d5a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:25.266059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2a150-2f63-4fbc-9a81-59a87328d5a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:25.266059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2a150-2f63-4fbc-9a81-59a87328d5a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:25.266059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2a150-2f63-4fbc-9a81-59a87328d5a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:25.333086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 813/1639] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:25 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:25.337974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c4e30-43dc-4434-87d9-b8d95df6b778 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:25.341470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c4e30-43dc-4434-87d9-b8d95df6b778 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:25.341470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c4e30-43dc-4434-87d9-b8d95df6b778 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:25.344057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2a150-2f63-4fbc-9a81-59a87328d5a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:25.348045 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2a150-2f63-4fbc-9a81-59a87328d5a4 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:25.351340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 814/1640] 10.222.0.86 () {54 vars in 912 bytes} [Thu Jul 28 18:12:25 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:12:25.352370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c4e30-43dc-4434-87d9-b8d95df6b778 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:25.364819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c4e30-43dc-4434-87d9-b8d95df6b778 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:25.368663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49afcce-a3c2-448f-9e13-fa930fd50a14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:25.369386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49afcce-a3c2-448f-9e13-fa930fd50a14 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:25.374937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49afcce-a3c2-448f-9e13-fa930fd50a14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:25.374937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49afcce-a3c2-448f-9e13-fa930fd50a14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:25.381070 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 827/1641] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:24 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:25.382682 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49afcce-a3c2-448f-9e13-fa930fd50a14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:25.382836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f68533-1d87-4268-8f48-a2cf9dee17c8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:25.382997 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f68533-1d87-4268-8f48-a2cf9dee17c8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:25.383252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f68533-1d87-4268-8f48-a2cf9dee17c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:25.384483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49afcce-a3c2-448f-9e13-fa930fd50a14 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:25.397435 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 815/1642] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:25 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:25.422190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-50f68533-1d87-4268-8f48-a2cf9dee17c8 None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:25.423413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-50f68533-1d87-4268-8f48-a2cf9dee17c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:25.455403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:25.543961 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49afcce-a3c2-448f-9e13-fa930fd50a14 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:25.546665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49afcce-a3c2-448f-9e13-fa930fd50a14 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:25.547880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 828/1643] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:12:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:25.553284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eec40-74e4-49da-b52a-92dae97ae842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:25.553924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eec40-74e4-49da-b52a-92dae97ae842 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:25.554260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eec40-74e4-49da-b52a-92dae97ae842 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:25.554541 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eec40-74e4-49da-b52a-92dae97ae842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:25.568111 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5eec40-74e4-49da-b52a-92dae97ae842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:25.569723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5eec40-74e4-49da-b52a-92dae97ae842 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:25.580472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 816/1644] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:25 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:12:25.697835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f68533-1d87-4268-8f48-a2cf9dee17c8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:25.722451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f68533-1d87-4268-8f48-a2cf9dee17c8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:25.730329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a222938-8344-45b5-b235-d7b96c571309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:25.732232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a222938-8344-45b5-b235-d7b96c571309 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:25.732522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a222938-8344-45b5-b235-d7b96c571309 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:25.737641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a222938-8344-45b5-b235-d7b96c571309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:25.789332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a222938-8344-45b5-b235-d7b96c571309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:25.791923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a222938-8344-45b5-b235-d7b96c571309 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:26.101670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 829/1645] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:26.288785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eec40-74e4-49da-b52a-92dae97ae842 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:26.298956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eec40-74e4-49da-b52a-92dae97ae842 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:26.305858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec389a-65f2-4a90-8ff4-d810d011f743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:26.306822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec389a-65f2-4a90-8ff4-d810d011f743 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:26.307443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec389a-65f2-4a90-8ff4-d810d011f743 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:26.307915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec389a-65f2-4a90-8ff4-d810d011f743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:26.378555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec389a-65f2-4a90-8ff4-d810d011f743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:26.380643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec389a-65f2-4a90-8ff4-d810d011f743 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:26.407760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 817/1646] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:26 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:26.769221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:27.187522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a222938-8344-45b5-b235-d7b96c571309 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:27.199867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a222938-8344-45b5-b235-d7b96c571309 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:27.216520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad01691-89cc-4dcb-8d23-02b6c97d064b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:27.216928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec389a-65f2-4a90-8ff4-d810d011f743 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:27.219850 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad01691-89cc-4dcb-8d23-02b6c97d064b service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:27.220464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad01691-89cc-4dcb-8d23-02b6c97d064b service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:27.220815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad01691-89cc-4dcb-8d23-02b6c97d064b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:27.283524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:27.365410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec389a-65f2-4a90-8ff4-d810d011f743 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:27.380010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:27.381190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:27.383096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:27.383530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] PATH_INFO: `/v3/users/7c22ee0181454cb9bd17e7fcbae27191` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:27.385540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c22ee0181454cb9bd17e7fcbae27191)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:27.389672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:12:27.389672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: warnings.warn(msg) Jul 28 18:12:27.389672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: Jul 28 18:12:27.391741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:27.446068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad01691-89cc-4dcb-8d23-02b6c97d064b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:27.449126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad01691-89cc-4dcb-8d23-02b6c97d064b service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:27.482984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 830/1647] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:27.671588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c22ee0181454cb9bd17e7fcbae27191'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:27.711505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c22ee0181454cb9bd17e7fcbae27191'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:27.805244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c22ee0181454cb9bd17e7fcbae27191'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:27.806663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c22ee0181454cb9bd17e7fcbae27191'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:27.825812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508adcd-a04e-4e5e-a732-42b56ee87990 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:27.826344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508adcd-a04e-4e5e-a732-42b56ee87990 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:27.827244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508adcd-a04e-4e5e-a732-42b56ee87990 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:27.829882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 831/1648] 10.222.0.86 () {54 vars in 962 bytes} [Thu Jul 28 18:12:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:12:27.836677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401da47d-1aaf-48ab-8314-791a5ab66b55 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:27.840414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401da47d-1aaf-48ab-8314-791a5ab66b55 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:27.840414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401da47d-1aaf-48ab-8314-791a5ab66b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:27.877095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f57737cf-04ac-4340-b815-8c9d58afe7cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c22ee0181454cb9bd17e7fcbae27191'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:27.893133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-401da47d-1aaf-48ab-8314-791a5ab66b55 None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:27.894973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-401da47d-1aaf-48ab-8314-791a5ab66b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:27.897457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 818/1649] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:27 2022] DELETE /identity/v3/users/7c22ee0181454cb9bd17e7fcbae27191 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:12:27.904814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77b6d-e64e-4f45-8774-cdbe23c90724 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:27.904814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77b6d-e64e-4f45-8774-cdbe23c90724 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:27.905071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77b6d-e64e-4f45-8774-cdbe23c90724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:27.972621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9ea77b6d-e64e-4f45-8774-cdbe23c90724 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:27.977572 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9ea77b6d-e64e-4f45-8774-cdbe23c90724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:28.028172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:28.069666 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 832/1650] 10.222.0.86 () {58 vars in 1059 bytes} [Thu Jul 28 18:12:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:28.074271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8080d-e8c6-4d1c-8591-eab30d48a62b None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:28.074744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8080d-e8c6-4d1c-8591-eab30d48a62b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:28.075437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8080d-e8c6-4d1c-8591-eab30d48a62b None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:28.077487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 833/1651] 10.222.0.86 () {54 vars in 912 bytes} [Thu Jul 28 18:12:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 18:12:28.085740 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7757cf-8895-4c3a-80fb-c1cd61bc689a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:28.086263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7757cf-8895-4c3a-80fb-c1cd61bc689a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:28.086741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7757cf-8895-4c3a-80fb-c1cd61bc689a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:28.119264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 819/1652] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:28.128102 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4d7757cf-8895-4c3a-80fb-c1cd61bc689a None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:28.129807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4d7757cf-8895-4c3a-80fb-c1cd61bc689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:28.141507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 834/1653] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:12:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:28.158294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea77b6d-e64e-4f45-8774-cdbe23c90724 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:28.161151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9ea77b6d-e64e-4f45-8774-cdbe23c90724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:28.186011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7757cf-8895-4c3a-80fb-c1cd61bc689a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:28.188621 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4d7757cf-8895-4c3a-80fb-c1cd61bc689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:28.267531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:28.343616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea77b6d-e64e-4f45-8774-cdbe23c90724 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:28.357259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90cd8fa-ea48-498c-9a29-86ec2b435ce9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:28.358144 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cd8fa-ea48-498c-9a29-86ec2b435ce9 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:28.358739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cd8fa-ea48-498c-9a29-86ec2b435ce9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:28.359948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cd8fa-ea48-498c-9a29-86ec2b435ce9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:28.364310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7757cf-8895-4c3a-80fb-c1cd61bc689a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:28.375256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b6c1d-6d68-439c-870c-48f45be926ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:28.376283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b6c1d-6d68-439c-870c-48f45be926ea service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:28.376647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b6c1d-6d68-439c-870c-48f45be926ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:28.378805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b6c1d-6d68-439c-870c-48f45be926ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:28.384585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90cd8fa-ea48-498c-9a29-86ec2b435ce9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:28.388271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90cd8fa-ea48-498c-9a29-86ec2b435ce9 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:28.404072 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b6c1d-6d68-439c-870c-48f45be926ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:28.407407 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b6c1d-6d68-439c-870c-48f45be926ea service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:28.451097 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 820/1654] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:28.466084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 835/1655] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:30.247086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90cd8fa-ea48-498c-9a29-86ec2b435ce9 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:30.257721 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b90cd8fa-ea48-498c-9a29-86ec2b435ce9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:30.296737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90cd8fa-ea48-498c-9a29-86ec2b435ce9 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:30.317996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:30.318951 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:30.319887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:30.320174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] PATH_INFO: `/v3/projects/c498ba33da3e47d68c406d75b01bf83a` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:30.324007 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c498ba33da3e47d68c406d75b01bf83a)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:30.330802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:30.537689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c498ba33da3e47d68c406d75b01bf83a', 'request_id': 'req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:30.556028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c498ba33da3e47d68c406d75b01bf83a', 'request_id': 'req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:30.637937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c498ba33da3e47d68c406d75b01bf83a', 'request_id': 'req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:30.664317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] The token cache is being invalidate because project c498ba33da3e47d68c406d75b01bf83a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:12:30.664629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:30.666211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-352475dd-4e53-4bb7-ae9f-206ddbf71eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:30.675413 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 821/1656] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:12:30 2022] DELETE /identity/v3/projects/c498ba33da3e47d68c406d75b01bf83a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:30.676635 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b6c1d-6d68-439c-870c-48f45be926ea service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:30.692019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b6c1d-6d68-439c-870c-48f45be926ea service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:30.848437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0523989-2a1e-4b56-8e53-e6502a5b244e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:30.849495 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0523989-2a1e-4b56-8e53-e6502a5b244e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:30.849940 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0523989-2a1e-4b56-8e53-e6502a5b244e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:30.850219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0523989-2a1e-4b56-8e53-e6502a5b244e admin admin] PATH_INFO: `/v3/users/02ee0257f9a44b35becc1422e2c5975a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:30.851500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0523989-2a1e-4b56-8e53-e6502a5b244e admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ee0257f9a44b35becc1422e2c5975a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:30.854348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0523989-2a1e-4b56-8e53-e6502a5b244e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:31.238140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a0523989-2a1e-4b56-8e53-e6502a5b244e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ee0257f9a44b35becc1422e2c5975a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:31.243280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a0523989-2a1e-4b56-8e53-e6502a5b244e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ee0257f9a44b35becc1422e2c5975a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:31.276121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a0523989-2a1e-4b56-8e53-e6502a5b244e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ee0257f9a44b35becc1422e2c5975a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:31.404483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a0523989-2a1e-4b56-8e53-e6502a5b244e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ee0257f9a44b35becc1422e2c5975a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:31.405328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a0523989-2a1e-4b56-8e53-e6502a5b244e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ee0257f9a44b35becc1422e2c5975a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:31.445197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:31.461720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 836/1657] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:30 2022] DELETE /identity/v3/users/02ee0257f9a44b35becc1422e2c5975a => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:31.602878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2696c46-f910-4d23-add5-dc915ffbe011 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:31.603897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2696c46-f910-4d23-add5-dc915ffbe011 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:31.604322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2696c46-f910-4d23-add5-dc915ffbe011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:31.654825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d2696c46-f910-4d23-add5-dc915ffbe011 None None] MFA Rules not processed for user `d96e9a9e40c54b6a9a7f8560e4846be5`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:31.656373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d2696c46-f910-4d23-add5-dc915ffbe011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:31.754090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:31.788116 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 822/1658] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:31.789836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd3997-6d82-405e-97e1-3530042d2365 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:31.790304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd3997-6d82-405e-97e1-3530042d2365 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:31.790402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd3997-6d82-405e-97e1-3530042d2365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:31.829947 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9ecd3997-6d82-405e-97e1-3530042d2365 None None] MFA Rules not processed for user `d96e9a9e40c54b6a9a7f8560e4846be5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:31.833377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9ecd3997-6d82-405e-97e1-3530042d2365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:31.859171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 837/1659] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:12:31.870942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2696c46-f910-4d23-add5-dc915ffbe011 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:31.873099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d2696c46-f910-4d23-add5-dc915ffbe011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:32.019287 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2696c46-f910-4d23-add5-dc915ffbe011 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:32.026964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206cc03-4e23-4f52-bbef-047a07254c33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:32.027625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206cc03-4e23-4f52-bbef-047a07254c33 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:32.027917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206cc03-4e23-4f52-bbef-047a07254c33 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:32.028192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206cc03-4e23-4f52-bbef-047a07254c33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:32.041326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206cc03-4e23-4f52-bbef-047a07254c33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:32.043039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206cc03-4e23-4f52-bbef-047a07254c33 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:32.058930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 823/1660] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:32.260820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd3997-6d82-405e-97e1-3530042d2365 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:32.262644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9ecd3997-6d82-405e-97e1-3530042d2365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:32.453651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd3997-6d82-405e-97e1-3530042d2365 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:32.491284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33058c4-2f3e-437d-ba22-8619b7b6470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:32.491782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33058c4-2f3e-437d-ba22-8619b7b6470b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:32.492344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33058c4-2f3e-437d-ba22-8619b7b6470b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:32.492927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33058c4-2f3e-437d-ba22-8619b7b6470b admin admin] PATH_INFO: `/v3/users/a4e1efaae5184880a180d4eda894426e` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:32.494350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33058c4-2f3e-437d-ba22-8619b7b6470b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e1efaae5184880a180d4eda894426e)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:32.504613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33058c4-2f3e-437d-ba22-8619b7b6470b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:32.735604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b33058c4-2f3e-437d-ba22-8619b7b6470b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e1efaae5184880a180d4eda894426e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:32.739678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b33058c4-2f3e-437d-ba22-8619b7b6470b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e1efaae5184880a180d4eda894426e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:32.771094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b33058c4-2f3e-437d-ba22-8619b7b6470b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e1efaae5184880a180d4eda894426e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:32.854151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b33058c4-2f3e-437d-ba22-8619b7b6470b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e1efaae5184880a180d4eda894426e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:32.854808 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b33058c4-2f3e-437d-ba22-8619b7b6470b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e1efaae5184880a180d4eda894426e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:32.885493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e18b9-1c3c-4234-a30e-700d874248ed None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:32.885876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e18b9-1c3c-4234-a30e-700d874248ed None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:32.885948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e18b9-1c3c-4234-a30e-700d874248ed None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:32.887548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 838/1661] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:32 2022] DELETE /identity/v3/users/a4e1efaae5184880a180d4eda894426e => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:32.887789 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 824/1662] 10.222.0.86 () {54 vars in 912 bytes} [Thu Jul 28 18:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 18:12:32.889095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563375e5-5fbd-4042-8db6-52b876f0511f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:32.889548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563375e5-5fbd-4042-8db6-52b876f0511f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:32.889905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563375e5-5fbd-4042-8db6-52b876f0511f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:32.904452 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93ac15-006b-465e-8bf9-e1ebdad21ca2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:32.904913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93ac15-006b-465e-8bf9-e1ebdad21ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:32.905877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93ac15-006b-465e-8bf9-e1ebdad21ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:32.927376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-563375e5-5fbd-4042-8db6-52b876f0511f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:32.928768 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-563375e5-5fbd-4042-8db6-52b876f0511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:32.988937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0a93ac15-006b-465e-8bf9-e1ebdad21ca2 None None] MFA Rules not processed for user `b038336582a0435dbd47ad2c09cd23e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:32.990563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0a93ac15-006b-465e-8bf9-e1ebdad21ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:33.043901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 839/1663] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:33.061010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563375e5-5fbd-4042-8db6-52b876f0511f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:33.062962 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-563375e5-5fbd-4042-8db6-52b876f0511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:33.115854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 825/1664] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:12:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:33.155648 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93ac15-006b-465e-8bf9-e1ebdad21ca2 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:33.157436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0a93ac15-006b-465e-8bf9-e1ebdad21ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:33.178761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93ac15-006b-465e-8bf9-e1ebdad21ca2 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:33.197442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592624f8-203f-44d9-a21e-95827435aafa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:33.198444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592624f8-203f-44d9-a21e-95827435aafa service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:33.198729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592624f8-203f-44d9-a21e-95827435aafa service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:33.199005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592624f8-203f-44d9-a21e-95827435aafa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:33.216429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563375e5-5fbd-4042-8db6-52b876f0511f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:33.218990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592624f8-203f-44d9-a21e-95827435aafa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:33.221210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592624f8-203f-44d9-a21e-95827435aafa service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:33.228865 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d8e83-a5f0-4093-829c-4adf1275c2f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:33.229548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d8e83-a5f0-4093-829c-4adf1275c2f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:33.229821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d8e83-a5f0-4093-829c-4adf1275c2f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:33.233271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d8e83-a5f0-4093-829c-4adf1275c2f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:33.236341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 826/1665] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:33.244602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d8e83-a5f0-4093-829c-4adf1275c2f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:33.246400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d8e83-a5f0-4093-829c-4adf1275c2f8 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:33.260654 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 840/1666] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:12:33.493042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592624f8-203f-44d9-a21e-95827435aafa service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:33.494650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-592624f8-203f-44d9-a21e-95827435aafa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:33.534021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592624f8-203f-44d9-a21e-95827435aafa service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:33.548501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:33.554998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:33.556533 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:33.556824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:33.557145 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] PATH_INFO: `/v3/projects/cb023f437dcc419c9be98f6036a14609` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:33.561850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb023f437dcc419c9be98f6036a14609)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:33.570432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:33.736867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb023f437dcc419c9be98f6036a14609', 'request_id': 'req-525d62fe-fe34-40fd-a919-767966920412'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:33.750382 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb023f437dcc419c9be98f6036a14609', 'request_id': 'req-525d62fe-fe34-40fd-a919-767966920412'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:33.769256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:33.831488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb023f437dcc419c9be98f6036a14609', 'request_id': 'req-525d62fe-fe34-40fd-a919-767966920412'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:33.849468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] The token cache is being invalidate because project cb023f437dcc419c9be98f6036a14609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:12:33.853476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:33.854571 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-525d62fe-fe34-40fd-a919-767966920412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:33.866080 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 827/1667] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:12:33 2022] DELETE /identity/v3/projects/cb023f437dcc419c9be98f6036a14609 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:33.871288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528568c5-1a23-4567-8ff6-303b32775916 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:33.871288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528568c5-1a23-4567-8ff6-303b32775916 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:33.871288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528568c5-1a23-4567-8ff6-303b32775916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:33.909101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-528568c5-1a23-4567-8ff6-303b32775916 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:33.913772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-528568c5-1a23-4567-8ff6-303b32775916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:34.052145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 841/1668] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:34.060983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c5795-4b24-42a9-bd09-92620864eed9 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:34.062683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c5795-4b24-42a9-bd09-92620864eed9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:34.063101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c5795-4b24-42a9-bd09-92620864eed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:34.104176 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-ee8c5795-4b24-42a9-bd09-92620864eed9 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:34.105583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ee8c5795-4b24-42a9-bd09-92620864eed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:34.116638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 828/1669] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:34.129445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528568c5-1a23-4567-8ff6-303b32775916 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:34.143367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528568c5-1a23-4567-8ff6-303b32775916 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:34.167973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6b9c6-e545-4653-b9a5-a38a48802c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:34.168672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6b9c6-e545-4653-b9a5-a38a48802c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:34.170244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6b9c6-e545-4653-b9a5-a38a48802c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:34.170537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6b9c6-e545-4653-b9a5-a38a48802c14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:34.172388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6b9c6-e545-4653-b9a5-a38a48802c14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:34.174864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6b9c6-e545-4653-b9a5-a38a48802c14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:34.176604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6b9c6-e545-4653-b9a5-a38a48802c14 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:34.215211 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 842/1670] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:34.220383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c5795-4b24-42a9-bd09-92620864eed9 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:34.231232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c5795-4b24-42a9-bd09-92620864eed9 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:34.248519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635a91d-c215-4e58-bb28-b4d93da67ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:34.252421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635a91d-c215-4e58-bb28-b4d93da67ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:34.252770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635a91d-c215-4e58-bb28-b4d93da67ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:34.256994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635a91d-c215-4e58-bb28-b4d93da67ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:34.257864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635a91d-c215-4e58-bb28-b4d93da67ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:34.259902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635a91d-c215-4e58-bb28-b4d93da67ca5 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:34.400680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 829/1671] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:34 2022] POST /identity/v3/projects => generated 371 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:34.405435 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6b9c6-e545-4653-b9a5-a38a48802c14 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:34.426493 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6b9c6-e545-4653-b9a5-a38a48802c14 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:34.552110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a2316-ee50-47eb-8e0b-2748e8ddb63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:34.552931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a2316-ee50-47eb-8e0b-2748e8ddb63f admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:34.553301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a2316-ee50-47eb-8e0b-2748e8ddb63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:34.553586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a2316-ee50-47eb-8e0b-2748e8ddb63f admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:34.553837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635a91d-c215-4e58-bb28-b4d93da67ca5 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:34.554270 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a2316-ee50-47eb-8e0b-2748e8ddb63f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:34.555683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f635a91d-c215-4e58-bb28-b4d93da67ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:34.556310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a2316-ee50-47eb-8e0b-2748e8ddb63f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:34.580252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635a91d-c215-4e58-bb28-b4d93da67ca5 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:34.590676 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:34.592186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:34.592476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:34.592806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] PATH_INFO: `/v3/projects/79a097f0698e4b0fa113b651a01051b7` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:34.594067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a097f0698e4b0fa113b651a01051b7)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:34.598954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:34.696649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 843/1672] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:34 2022] POST /identity/v3/users => generated 286 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:34.703752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a2316-ee50-47eb-8e0b-2748e8ddb63f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:34.755362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a2316-ee50-47eb-8e0b-2748e8ddb63f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:34.779873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a097f0698e4b0fa113b651a01051b7', 'request_id': 'req-18ce6843-da28-43bb-9a3f-7c9a303e5852'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:34.806591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a097f0698e4b0fa113b651a01051b7', 'request_id': 'req-18ce6843-da28-43bb-9a3f-7c9a303e5852'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:34.877534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a097f0698e4b0fa113b651a01051b7', 'request_id': 'req-18ce6843-da28-43bb-9a3f-7c9a303e5852'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:34.904655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] The token cache is being invalidate because project 79a097f0698e4b0fa113b651a01051b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:12:34.904891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:34.906587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-18ce6843-da28-43bb-9a3f-7c9a303e5852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:34.907923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c1d41-f0bb-47a5-9900-a8408623e60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:34.908685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c1d41-f0bb-47a5-9900-a8408623e60f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:34.914499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c1d41-f0bb-47a5-9900-a8408623e60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:34.914499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c1d41-f0bb-47a5-9900-a8408623e60f admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:34.914499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c1d41-f0bb-47a5-9900-a8408623e60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:34.914730 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 830/1673] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:12:34 2022] DELETE /identity/v3/projects/79a097f0698e4b0fa113b651a01051b7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:12:34.914813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c1d41-f0bb-47a5-9900-a8408623e60f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:34.914908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bc488-1039-430c-b8ac-d8d1ae378d81 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:34.914908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bc488-1039-430c-b8ac-d8d1ae378d81 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:34.914908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bc488-1039-430c-b8ac-d8d1ae378d81 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:34.916523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 831/1674] 10.222.0.86 () {54 vars in 961 bytes} [Thu Jul 28 18:12:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:12:34.918112 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a587852-3fca-4620-a594-51eceba83edf None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:34.918406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a587852-3fca-4620-a594-51eceba83edf None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:34.918715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a587852-3fca-4620-a594-51eceba83edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:34.937626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 844/1675] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:34.939326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adefe2-461c-4ef4-b277-2c6ac8f067f8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:34.939721 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adefe2-461c-4ef4-b277-2c6ac8f067f8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:34.940109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adefe2-461c-4ef4-b277-2c6ac8f067f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:34.949020 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-4a587852-3fca-4620-a594-51eceba83edf None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:34.950407 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4a587852-3fca-4620-a594-51eceba83edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:34.982977 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 832/1676] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:34.984762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a587852-3fca-4620-a594-51eceba83edf None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:34.989833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e5adefe2-461c-4ef4-b277-2c6ac8f067f8 None None] MFA Rules not processed for user `b20564a83953433290aa3fabf884370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:34.990462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4a587852-3fca-4620-a594-51eceba83edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:34.991541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e5adefe2-461c-4ef4-b277-2c6ac8f067f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:35.016569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a587852-3fca-4620-a594-51eceba83edf None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:35.027148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2576b3-aec3-483b-bb93-72e00103a404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:35.028111 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2576b3-aec3-483b-bb93-72e00103a404 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:35.028497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2576b3-aec3-483b-bb93-72e00103a404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:35.029121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2576b3-aec3-483b-bb93-72e00103a404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:35.030521 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2576b3-aec3-483b-bb93-72e00103a404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:35.035819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2576b3-aec3-483b-bb93-72e00103a404 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:35.060030 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 833/1677] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:35.061933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e7274-6653-4891-b301-fe6f1a362426 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:35.062437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e7274-6653-4891-b301-fe6f1a362426 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:35.063035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e7274-6653-4891-b301-fe6f1a362426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:35.147827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0e2e7274-6653-4891-b301-fe6f1a362426 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:35.152177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0e2e7274-6653-4891-b301-fe6f1a362426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:35.179476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 834/1678] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:35.180336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e7274-6653-4891-b301-fe6f1a362426 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:35.191408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 845/1679] 10.222.0.86 () {58 vars in 1058 bytes} [Thu Jul 28 18:12:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:35.192093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5adefe2-461c-4ef4-b277-2c6ac8f067f8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:35.207557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e7274-6653-4891-b301-fe6f1a362426 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:35.215325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5adefe2-461c-4ef4-b277-2c6ac8f067f8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:35.220350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1525b-70e4-44ca-b464-7fdba4738a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:35.221668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1525b-70e4-44ca-b464-7fdba4738a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:35.222428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1525b-70e4-44ca-b464-7fdba4738a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:35.231226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1525b-70e4-44ca-b464-7fdba4738a1f admin admin] PATH_INFO: `/v3/projects/bb2a698a9b6a4d9a9b67a93685997f21/users/3b5ce0d3338e43cba384cf26debd1a2a/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:35.231350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510772eb-c552-4966-9540-da77ad4db7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:35.231441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510772eb-c552-4966-9540-da77ad4db7db admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:35.231651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510772eb-c552-4966-9540-da77ad4db7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:35.231929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510772eb-c552-4966-9540-da77ad4db7db admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:35.232678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510772eb-c552-4966-9540-da77ad4db7db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:35.233006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510772eb-c552-4966-9540-da77ad4db7db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:35.240818 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510772eb-c552-4966-9540-da77ad4db7db admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:35.263621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1525b-70e4-44ca-b464-7fdba4738a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2a698a9b6a4d9a9b67a93685997f21, user_id=3b5ce0d3338e43cba384cf26debd1a2a, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:35.271395 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1525b-70e4-44ca-b464-7fdba4738a1f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:35.275900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 846/1680] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:35.277084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510772eb-c552-4966-9540-da77ad4db7db admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:35.303564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510772eb-c552-4966-9540-da77ad4db7db admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:35.323001 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9770109-cd1a-4303-b10e-db7e3264a755 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:35.323172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9770109-cd1a-4303-b10e-db7e3264a755 service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:35.323408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9770109-cd1a-4303-b10e-db7e3264a755 service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:35.323663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9770109-cd1a-4303-b10e-db7e3264a755 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:35.327697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f9770109-cd1a-4303-b10e-db7e3264a755 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:35.348223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 835/1681] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:35 2022] PUT /identity/v3/projects/bb2a698a9b6a4d9a9b67a93685997f21/users/3b5ce0d3338e43cba384cf26debd1a2a/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:35.349732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1525b-70e4-44ca-b464-7fdba4738a1f admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:35.390898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9770109-cd1a-4303-b10e-db7e3264a755 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:35.395963 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9770109-cd1a-4303-b10e-db7e3264a755 service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:35.428490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:35.447839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 847/1682] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:35.455066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104222-9d16-4881-9266-98d6718ddfb9 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:35.455445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104222-9d16-4881-9266-98d6718ddfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:35.458082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104222-9d16-4881-9266-98d6718ddfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:35.522808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:35.524308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8f104222-9d16-4881-9266-98d6718ddfb9 None None] MFA Rules not processed for user `3b5ce0d3338e43cba384cf26debd1a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:35.525571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8f104222-9d16-4881-9266-98d6718ddfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:35.552994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1525b-70e4-44ca-b464-7fdba4738a1f admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:35.584081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437448e-885f-4953-a109-6d71249ada89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:35.584761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437448e-885f-4953-a109-6d71249ada89 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:35.585065 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437448e-885f-4953-a109-6d71249ada89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:35.585346 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437448e-885f-4953-a109-6d71249ada89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:35.586178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437448e-885f-4953-a109-6d71249ada89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:35.588222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437448e-885f-4953-a109-6d71249ada89 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:35.676169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 848/1683] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:35.678061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f104222-9d16-4881-9266-98d6718ddfb9 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:35.693320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f104222-9d16-4881-9266-98d6718ddfb9 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:35.705824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ab39b-f4f2-4fda-8281-dc601ac084c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:35.706784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ab39b-f4f2-4fda-8281-dc601ac084c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:35.707822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ab39b-f4f2-4fda-8281-dc601ac084c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:35.708844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ab39b-f4f2-4fda-8281-dc601ac084c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:35.711365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ab39b-f4f2-4fda-8281-dc601ac084c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:35.716096 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ab39b-f4f2-4fda-8281-dc601ac084c0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:35.761614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 836/1684] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:35 2022] POST /identity/v3/projects => generated 371 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:35.766106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437448e-885f-4953-a109-6d71249ada89 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:35.902268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 849/1685] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:35 2022] POST /identity/v3/projects => generated 373 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:12:35.906891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437448e-885f-4953-a109-6d71249ada89 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:35.907428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ab39b-f4f2-4fda-8281-dc601ac084c0 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:36.009239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00cf89-e1fa-4b2b-a879-34fc9b23c3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:36.010348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00cf89-e1fa-4b2b-a879-34fc9b23c3df admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:36.011472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00cf89-e1fa-4b2b-a879-34fc9b23c3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:36.011930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00cf89-e1fa-4b2b-a879-34fc9b23c3df admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:36.012819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00cf89-e1fa-4b2b-a879-34fc9b23c3df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:36.026192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00cf89-e1fa-4b2b-a879-34fc9b23c3df admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:36.095479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ab39b-f4f2-4fda-8281-dc601ac084c0 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:36.108156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141af7c-85e7-446c-a11f-182d341cfd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:36.111051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141af7c-85e7-446c-a11f-182d341cfd4c admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:36.111420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141af7c-85e7-446c-a11f-182d341cfd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:36.111706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141af7c-85e7-446c-a11f-182d341cfd4c admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:36.112440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141af7c-85e7-446c-a11f-182d341cfd4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:36.115040 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141af7c-85e7-446c-a11f-182d341cfd4c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:36.119539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 837/1686] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:35 2022] POST /identity/v3/users => generated 286 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:12:36.121010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00cf89-e1fa-4b2b-a879-34fc9b23c3df admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:36.125157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-eb00cf89-e1fa-4b2b-a879-34fc9b23c3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:36.143833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00cf89-e1fa-4b2b-a879-34fc9b23c3df admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:36.150120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:36.151599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:36.151599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:36.151599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] PATH_INFO: `/v3/users/664cbcf779a84c3199b38770f3aa5848` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:36.156277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=664cbcf779a84c3199b38770f3aa5848)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:36.156277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:36.277823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 850/1687] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:35 2022] POST /identity/v3/users => generated 293 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:36.278824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141af7c-85e7-446c-a11f-182d341cfd4c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:36.302666 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141af7c-85e7-446c-a11f-182d341cfd4c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:36.317155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b801415-1aea-4246-b352-ca22aff2bf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:36.317870 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b801415-1aea-4246-b352-ca22aff2bf2b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:36.318155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b801415-1aea-4246-b352-ca22aff2bf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:36.318434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b801415-1aea-4246-b352-ca22aff2bf2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:36.319193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b801415-1aea-4246-b352-ca22aff2bf2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:36.322125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b801415-1aea-4246-b352-ca22aff2bf2b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:36.349807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 851/1688] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:36.350799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b801415-1aea-4246-b352-ca22aff2bf2b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:36.380515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b801415-1aea-4246-b352-ca22aff2bf2b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:36.382339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '664cbcf779a84c3199b38770f3aa5848'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:36.400604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '664cbcf779a84c3199b38770f3aa5848'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:36.404139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33402526-d830-462a-bc21-9bbadff7c7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:36.404785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402526-d830-462a-bc21-9bbadff7c7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:36.405197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402526-d830-462a-bc21-9bbadff7c7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:36.405484 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402526-d830-462a-bc21-9bbadff7c7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:36.406204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33402526-d830-462a-bc21-9bbadff7c7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:36.408638 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33402526-d830-462a-bc21-9bbadff7c7b3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:36.437209 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 852/1689] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:36.438438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33402526-d830-462a-bc21-9bbadff7c7b3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:36.454619 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33402526-d830-462a-bc21-9bbadff7c7b3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:36.463127 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b7c61-5c45-433f-9dc4-04866ec169af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:36.463936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7c61-5c45-433f-9dc4-04866ec169af admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:36.465051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7c61-5c45-433f-9dc4-04866ec169af admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:36.465051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7c61-5c45-433f-9dc4-04866ec169af admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:36.469075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b7c61-5c45-433f-9dc4-04866ec169af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:36.469075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b7c61-5c45-433f-9dc4-04866ec169af admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:36.485163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 853/1690] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:36.486376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b7c61-5c45-433f-9dc4-04866ec169af admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:36.503063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b7c61-5c45-433f-9dc4-04866ec169af admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:36.509655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff87aec-98f7-4dc9-ab9b-8672ab00041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:36.510660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87aec-98f7-4dc9-ab9b-8672ab00041e admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:36.511411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87aec-98f7-4dc9-ab9b-8672ab00041e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:36.511834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87aec-98f7-4dc9-ab9b-8672ab00041e admin admin] PATH_INFO: `/v3/projects/d80959bc029e4d539a67e10637702a86/users/684b7940558644d686d59f86967add68/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:36.528569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '664cbcf779a84c3199b38770f3aa5848'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:36.529248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '664cbcf779a84c3199b38770f3aa5848'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:36.542227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff87aec-98f7-4dc9-ab9b-8672ab00041e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d80959bc029e4d539a67e10637702a86, user_id=684b7940558644d686d59f86967add68, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:36.545306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff87aec-98f7-4dc9-ab9b-8672ab00041e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:36.555954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '664cbcf779a84c3199b38770f3aa5848'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:36.569598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 838/1691] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:36 2022] DELETE /identity/v3/users/664cbcf779a84c3199b38770f3aa5848 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:36.574252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:36.578471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:36.634522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 854/1692] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:36 2022] PUT /identity/v3/projects/d80959bc029e4d539a67e10637702a86/users/684b7940558644d686d59f86967add68/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:36.635626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c8c01-3f30-45ba-aeb7-813cdfd95664 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:36.635929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c8c01-3f30-45ba-aeb7-813cdfd95664 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:36.636212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c8c01-3f30-45ba-aeb7-813cdfd95664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:36.683759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ef2c8c01-3f30-45ba-aeb7-813cdfd95664 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:36.685188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ef2c8c01-3f30-45ba-aeb7-813cdfd95664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:36.784281 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62656fa9-9c25-4245-b502-08c2e2b8c9ae admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:36.856946 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 855/1693] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:36.858721 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c8c01-3f30-45ba-aeb7-813cdfd95664 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:36.861701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ef2c8c01-3f30-45ba-aeb7-813cdfd95664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:36.894040 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c8c01-3f30-45ba-aeb7-813cdfd95664 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:36.906297 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffbde6-e6ae-4881-a48d-01cffddfa6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:36.907232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffbde6-e6ae-4881-a48d-01cffddfa6ee admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:36.907519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffbde6-e6ae-4881-a48d-01cffddfa6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:36.907794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffbde6-e6ae-4881-a48d-01cffddfa6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:36.909804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffbde6-e6ae-4881-a48d-01cffddfa6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:36.930536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffbde6-e6ae-4881-a48d-01cffddfa6ee admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:36.955060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6544c96-bb51-4fa9-b6a2-1c0169687e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:36.955940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6544c96-bb51-4fa9-b6a2-1c0169687e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:36.957707 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6544c96-bb51-4fa9-b6a2-1c0169687e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:36.959012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6544c96-bb51-4fa9-b6a2-1c0169687e56 admin admin] PATH_INFO: `/v3/projects/95965f070ef34af7954d275e57a21307/users/c3c41ba1c05549368d858b7932ef5e8d/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:36.962126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 856/1694] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:36.963312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffbde6-e6ae-4881-a48d-01cffddfa6ee admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:36.971638 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f1ffbde6-e6ae-4881-a48d-01cffddfa6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:37.006420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6544c96-bb51-4fa9-b6a2-1c0169687e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95965f070ef34af7954d275e57a21307, user_id=c3c41ba1c05549368d858b7932ef5e8d, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:37.017393 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6544c96-bb51-4fa9-b6a2-1c0169687e56 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:37.125977 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 839/1695] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:36 2022] PUT /identity/v3/projects/95965f070ef34af7954d275e57a21307/users/c3c41ba1c05549368d858b7932ef5e8d/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:37.126649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6544c96-bb51-4fa9-b6a2-1c0169687e56 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:37.161569 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffbde6-e6ae-4881-a48d-01cffddfa6ee admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:37.263677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6544c96-bb51-4fa9-b6a2-1c0169687e56 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:37.269919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:37.308448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972ba19-4eff-41db-b156-85a8f5e2a450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:37.314871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972ba19-4eff-41db-b156-85a8f5e2a450 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:37.314871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972ba19-4eff-41db-b156-85a8f5e2a450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:37.314871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972ba19-4eff-41db-b156-85a8f5e2a450 admin admin] PATH_INFO: `/v3/projects/d80959bc029e4d539a67e10637702a86/users/684b7940558644d686d59f86967add68/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:37.319292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908899e-8d6f-440f-858e-56fb3dd38955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:37.321561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908899e-8d6f-440f-858e-56fb3dd38955 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:37.322681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972ba19-4eff-41db-b156-85a8f5e2a450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d80959bc029e4d539a67e10637702a86, user_id=684b7940558644d686d59f86967add68, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:37.323538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908899e-8d6f-440f-858e-56fb3dd38955 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:37.323942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908899e-8d6f-440f-858e-56fb3dd38955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:37.364670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972ba19-4eff-41db-b156-85a8f5e2a450 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:37.383995 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908899e-8d6f-440f-858e-56fb3dd38955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:37.383995 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908899e-8d6f-440f-858e-56fb3dd38955 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:37.448685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 840/1696] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:37 2022] PUT /identity/v3/projects/d80959bc029e4d539a67e10637702a86/users/684b7940558644d686d59f86967add68/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:37.451353 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa1664-2679-4617-aeea-2cad0a7bedc7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:37.457475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa1664-2679-4617-aeea-2cad0a7bedc7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:37.458109 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa1664-2679-4617-aeea-2cad0a7bedc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:37.480268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 857/1697] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:37.481951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c0ef5-4bc4-48f0-9058-16cd18ffdd65 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:37.482244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c0ef5-4bc4-48f0-9058-16cd18ffdd65 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:37.484198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c0ef5-4bc4-48f0-9058-16cd18ffdd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:37.535580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-22fa1664-2679-4617-aeea-2cad0a7bedc7 None None] MFA Rules not processed for user `c3c41ba1c05549368d858b7932ef5e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:37.540072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-22fa1664-2679-4617-aeea-2cad0a7bedc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:37.553138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ca1c0ef5-4bc4-48f0-9058-16cd18ffdd65 None None] MFA Rules not processed for user `684b7940558644d686d59f86967add68`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:37.554517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ca1c0ef5-4bc4-48f0-9058-16cd18ffdd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:37.709300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 841/1698] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:37.711343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa1664-2679-4617-aeea-2cad0a7bedc7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:37.735868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 858/1699] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:37.749730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c0ef5-4bc4-48f0-9058-16cd18ffdd65 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:37.751712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ca1c0ef5-4bc4-48f0-9058-16cd18ffdd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:37.808202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa1664-2679-4617-aeea-2cad0a7bedc7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:37.815962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fc43d-094d-47f2-b5d2-2638b3d04695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:37.829946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fc43d-094d-47f2-b5d2-2638b3d04695 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:37.835461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fc43d-094d-47f2-b5d2-2638b3d04695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:37.835461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fc43d-094d-47f2-b5d2-2638b3d04695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:37.835461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fc43d-094d-47f2-b5d2-2638b3d04695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:37.835461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fc43d-094d-47f2-b5d2-2638b3d04695 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:37.919786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c0ef5-4bc4-48f0-9058-16cd18ffdd65 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:37.931344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae901da-c79c-4573-b953-fafeab84d4c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:37.931704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae901da-c79c-4573-b953-fafeab84d4c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:37.931991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae901da-c79c-4573-b953-fafeab84d4c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:37.932262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae901da-c79c-4573-b953-fafeab84d4c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:37.954105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae901da-c79c-4573-b953-fafeab84d4c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:37.956658 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae901da-c79c-4573-b953-fafeab84d4c4 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:37.964231 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 842/1700] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:37 2022] POST /identity/v3/projects => generated 373 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:37.971555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fc43d-094d-47f2-b5d2-2638b3d04695 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:37.985044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 859/1701] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:12:38.170302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fc43d-094d-47f2-b5d2-2638b3d04695 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:38.177143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4c03b-610a-44f9-b044-229a226caf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:38.178030 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4c03b-610a-44f9-b044-229a226caf45 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:38.178748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4c03b-610a-44f9-b044-229a226caf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:38.179159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4c03b-610a-44f9-b044-229a226caf45 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:38.180064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4c03b-610a-44f9-b044-229a226caf45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:38.182354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4c03b-610a-44f9-b044-229a226caf45 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:38.324810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 843/1702] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:37 2022] POST /identity/v3/users => generated 293 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:38.329555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae901da-c79c-4573-b953-fafeab84d4c4 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:38.345098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae901da-c79c-4573-b953-fafeab84d4c4 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:38.356118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e9e93-7000-4bf0-af17-0b3280653af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:38.359628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e9e93-7000-4bf0-af17-0b3280653af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:38.360099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e9e93-7000-4bf0-af17-0b3280653af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:38.360588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e9e93-7000-4bf0-af17-0b3280653af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:38.361545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e9e93-7000-4bf0-af17-0b3280653af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:38.363388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e9e93-7000-4bf0-af17-0b3280653af3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:38.406708 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 860/1703] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:38.411501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4c03b-610a-44f9-b044-229a226caf45 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:38.448150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4c03b-610a-44f9-b044-229a226caf45 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:38.477010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690933b8-0da7-45ec-b347-ae1dbd051d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:38.484091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690933b8-0da7-45ec-b347-ae1dbd051d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:38.488911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690933b8-0da7-45ec-b347-ae1dbd051d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:38.489203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690933b8-0da7-45ec-b347-ae1dbd051d59 admin admin] PATH_INFO: `/v3/projects/daa113f206fb4715ac8918f76a385ba9/users/a55e5d2cbcda4be79b24058696d72958/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:38.552631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690933b8-0da7-45ec-b347-ae1dbd051d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa113f206fb4715ac8918f76a385ba9, user_id=a55e5d2cbcda4be79b24058696d72958, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:38.557828 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690933b8-0da7-45ec-b347-ae1dbd051d59 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:38.691630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e9e93-7000-4bf0-af17-0b3280653af3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:38.699134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 844/1704] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:38 2022] PUT /identity/v3/projects/daa113f206fb4715ac8918f76a385ba9/users/a55e5d2cbcda4be79b24058696d72958/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:38.709429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e9e93-7000-4bf0-af17-0b3280653af3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:38.826350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:39.013930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35deeed5-60f6-443e-8b6a-ba803b1782c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:39.024139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35deeed5-60f6-443e-8b6a-ba803b1782c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:39.029851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35deeed5-60f6-443e-8b6a-ba803b1782c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:39.029851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35deeed5-60f6-443e-8b6a-ba803b1782c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:39.029851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35deeed5-60f6-443e-8b6a-ba803b1782c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:39.043998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35deeed5-60f6-443e-8b6a-ba803b1782c7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:39.108265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690933b8-0da7-45ec-b347-ae1dbd051d59 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:39.109177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 861/1705] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:12:39.133180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690933b8-0da7-45ec-b347-ae1dbd051d59 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:39.151538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff6c52-3226-4680-bb31-c23c97acb022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:39.153183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff6c52-3226-4680-bb31-c23c97acb022 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:39.153472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff6c52-3226-4680-bb31-c23c97acb022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:39.153749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff6c52-3226-4680-bb31-c23c97acb022 admin admin] PATH_INFO: `/v3/projects/daa113f206fb4715ac8918f76a385ba9/users/a55e5d2cbcda4be79b24058696d72958/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:39.171246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff6c52-3226-4680-bb31-c23c97acb022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa113f206fb4715ac8918f76a385ba9, user_id=a55e5d2cbcda4be79b24058696d72958, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:39.217383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff6c52-3226-4680-bb31-c23c97acb022 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:39.274429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 845/1706] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:39 2022] PUT /identity/v3/projects/daa113f206fb4715ac8918f76a385ba9/users/a55e5d2cbcda4be79b24058696d72958/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:12:39.285685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9acd62-cb4b-412d-a01f-26b46b1d109d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:39.285980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9acd62-cb4b-412d-a01f-26b46b1d109d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:39.286267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9acd62-cb4b-412d-a01f-26b46b1d109d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:39.366268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4f9acd62-cb4b-412d-a01f-26b46b1d109d None None] MFA Rules not processed for user `a55e5d2cbcda4be79b24058696d72958`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:39.367519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4f9acd62-cb4b-412d-a01f-26b46b1d109d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:39.449019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:39.653081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 862/1707] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:39.663909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff6c52-3226-4680-bb31-c23c97acb022 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:39.665992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b8ff6c52-3226-4680-bb31-c23c97acb022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:39.816058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff6c52-3226-4680-bb31-c23c97acb022 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:39.825075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a29be-a255-4c98-af13-7ecbf6cb34c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:39.828100 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a29be-a255-4c98-af13-7ecbf6cb34c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:39.828514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a29be-a255-4c98-af13-7ecbf6cb34c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:39.829325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a29be-a255-4c98-af13-7ecbf6cb34c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:39.852291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a29be-a255-4c98-af13-7ecbf6cb34c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:39.856638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a29be-a255-4c98-af13-7ecbf6cb34c6 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:39.895159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9acd62-cb4b-412d-a01f-26b46b1d109d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:39.901826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 846/1708] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:40.019490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd7b2b-5dd8-4e08-9db5-609f9c85f43b None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:40.019896 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd7b2b-5dd8-4e08-9db5-609f9c85f43b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:40.020291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd7b2b-5dd8-4e08-9db5-609f9c85f43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:40.075239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9acd62-cb4b-412d-a01f-26b46b1d109d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:40.081800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:40.087572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:40.087572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:40.087572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] PATH_INFO: `/v3/users/3b5ce0d3338e43cba384cf26debd1a2a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:40.087572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b5ce0d3338e43cba384cf26debd1a2a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:40.089113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:40.122015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e0fd7b2b-5dd8-4e08-9db5-609f9c85f43b None None] MFA Rules not processed for user `ca7d773290a9483d91ebefad341addee`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:40.129760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e0fd7b2b-5dd8-4e08-9db5-609f9c85f43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:40.329697 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 847/1709] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:40.330913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd7b2b-5dd8-4e08-9db5-609f9c85f43b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:40.333846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e0fd7b2b-5dd8-4e08-9db5-609f9c85f43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:40.357315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd7b2b-5dd8-4e08-9db5-609f9c85f43b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:40.363938 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b5ce0d3338e43cba384cf26debd1a2a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:40.364679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b5ce0d3338e43cba384cf26debd1a2a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:40.368692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:40.372222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:40.379450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:40.380107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] PATH_INFO: `/v3/projects/128b148a8b70445d907e6f8433411929` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:40.384693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=128b148a8b70445d907e6f8433411929)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:40.384693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:40.412659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b5ce0d3338e43cba384cf26debd1a2a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:40.534404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b5ce0d3338e43cba384cf26debd1a2a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:40.535090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b5ce0d3338e43cba384cf26debd1a2a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:40.572832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 863/1710] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:39 2022] DELETE /identity/v3/users/3b5ce0d3338e43cba384cf26debd1a2a => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:40.573846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:40.577548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:40.617728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '128b148a8b70445d907e6f8433411929', 'request_id': 'req-b0032400-6418-4b2a-a8b1-100b209dba21'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:40.638005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '128b148a8b70445d907e6f8433411929', 'request_id': 'req-b0032400-6418-4b2a-a8b1-100b209dba21'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:40.711205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '128b148a8b70445d907e6f8433411929', 'request_id': 'req-b0032400-6418-4b2a-a8b1-100b209dba21'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:40.730670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc215da-43a8-40a1-b7c5-ef9ab8780e5a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:40.746299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25c639-862e-46d5-9d4f-10852a11eae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:40.747394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25c639-862e-46d5-9d4f-10852a11eae4 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:40.754776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25c639-862e-46d5-9d4f-10852a11eae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:40.754776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25c639-862e-46d5-9d4f-10852a11eae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:40.754776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5f25c639-862e-46d5-9d4f-10852a11eae4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:40.755126 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] The token cache is being invalidate because project 128b148a8b70445d907e6f8433411929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:12:40.770098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:40.772612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:40.778133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 848/1711] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:12:40 2022] DELETE /identity/v3/projects/128b148a8b70445d907e6f8433411929 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:40.779075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:40.785978 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:40.898617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:40.917198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25c639-862e-46d5-9d4f-10852a11eae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:40.918678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25c639-862e-46d5-9d4f-10852a11eae4 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:40.946120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 864/1712] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:40.948037 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c741587-8455-4561-a818-89ef6e1710e8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:40.948468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c741587-8455-4561-a818-89ef6e1710e8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:40.949913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c741587-8455-4561-a818-89ef6e1710e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:41.008410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0032400-6418-4b2a-a8b1-100b209dba21 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:41.011454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8c741587-8455-4561-a818-89ef6e1710e8 None None] MFA Rules not processed for user `b9874df18af245688db1137b6f1b0bff`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:41.012946 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8c741587-8455-4561-a818-89ef6e1710e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:41.193462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7378346-3d5c-4658-9c9d-152fd4722bc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:41.194403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7378346-3d5c-4658-9c9d-152fd4722bc6 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:41.194922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7378346-3d5c-4658-9c9d-152fd4722bc6 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:41.195462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7378346-3d5c-4658-9c9d-152fd4722bc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:41.208055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d7378346-3d5c-4658-9c9d-152fd4722bc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:41.214033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 865/1713] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:41.219826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5abfa-45c2-43f1-b433-2d03a12274f6 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:41.222800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5abfa-45c2-43f1-b433-2d03a12274f6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:41.223268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5abfa-45c2-43f1-b433-2d03a12274f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:41.294276 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-fea5abfa-45c2-43f1-b433-2d03a12274f6 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:41.295630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fea5abfa-45c2-43f1-b433-2d03a12274f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:41.433834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7378346-3d5c-4658-9c9d-152fd4722bc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:41.437342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7378346-3d5c-4658-9c9d-152fd4722bc6 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:41.494503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 849/1714] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:41.496403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7378346-3d5c-4658-9c9d-152fd4722bc6 service nova] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:41.509656 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d7378346-3d5c-4658-9c9d-152fd4722bc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:41.525288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 866/1715] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:41.526564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5abfa-45c2-43f1-b433-2d03a12274f6 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:41.539029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fea5abfa-45c2-43f1-b433-2d03a12274f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:41.552529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:41.574952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7378346-3d5c-4658-9c9d-152fd4722bc6 service nova] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:41.601734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4abd0-68cf-49c6-883f-1f65c73beddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:41.602745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4abd0-68cf-49c6-883f-1f65c73beddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:41.603213 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4abd0-68cf-49c6-883f-1f65c73beddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:41.603684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4abd0-68cf-49c6-883f-1f65c73beddf admin admin] PATH_INFO: `/v3/users/30dbf3429b2143b8882435cd0ff718e5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:41.605009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4abd0-68cf-49c6-883f-1f65c73beddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=30dbf3429b2143b8882435cd0ff718e5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:41.606956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4abd0-68cf-49c6-883f-1f65c73beddf admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:41.614812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5abfa-45c2-43f1-b433-2d03a12274f6 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:41.647595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424f2fa-4cf6-4148-b5fa-af1725c3c6b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:41.648286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424f2fa-4cf6-4148-b5fa-af1725c3c6b3 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:41.648518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424f2fa-4cf6-4148-b5fa-af1725c3c6b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:41.648918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424f2fa-4cf6-4148-b5fa-af1725c3c6b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:41.666681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424f2fa-4cf6-4148-b5fa-af1725c3c6b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:41.669428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424f2fa-4cf6-4148-b5fa-af1725c3c6b3 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:41.679221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:41.709263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 867/1716] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:41.710773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76f80c-f729-4b43-bf61-f70cb0d105d0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:41.711064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76f80c-f729-4b43-bf61-f70cb0d105d0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:41.711345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76f80c-f729-4b43-bf61-f70cb0d105d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:41.877046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7a76f80c-f729-4b43-bf61-f70cb0d105d0 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:41.880054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7a76f80c-f729-4b43-bf61-f70cb0d105d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:41.922022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f0a4abd0-68cf-49c6-883f-1f65c73beddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30dbf3429b2143b8882435cd0ff718e5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:41.945281 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f0a4abd0-68cf-49c6-883f-1f65c73beddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30dbf3429b2143b8882435cd0ff718e5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:42.047038 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 868/1717] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:12:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:42.052938 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76f80c-f729-4b43-bf61-f70cb0d105d0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:42.055140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7a76f80c-f729-4b43-bf61-f70cb0d105d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:42.060299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f0a4abd0-68cf-49c6-883f-1f65c73beddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30dbf3429b2143b8882435cd0ff718e5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:42.061823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f0a4abd0-68cf-49c6-883f-1f65c73beddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30dbf3429b2143b8882435cd0ff718e5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:42.083397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f0a4abd0-68cf-49c6-883f-1f65c73beddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30dbf3429b2143b8882435cd0ff718e5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:42.105825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 850/1718] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:41 2022] DELETE /identity/v3/users/30dbf3429b2143b8882435cd0ff718e5 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:42.213234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76f80c-f729-4b43-bf61-f70cb0d105d0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:42.309215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b271c3-649e-4340-9bdc-d2fca1296e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:42.309872 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b271c3-649e-4340-9bdc-d2fca1296e8c service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:42.310158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b271c3-649e-4340-9bdc-d2fca1296e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:42.310441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b271c3-649e-4340-9bdc-d2fca1296e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:42.312990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c0b271c3-649e-4340-9bdc-d2fca1296e8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:42.418307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b271c3-649e-4340-9bdc-d2fca1296e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:42.420031 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b271c3-649e-4340-9bdc-d2fca1296e8c service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:42.433925 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 869/1719] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:42.536722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426e23e-5e8c-471a-973b-6af594be21a8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:42.537328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426e23e-5e8c-471a-973b-6af594be21a8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:42.537422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426e23e-5e8c-471a-973b-6af594be21a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:42.574320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b271c3-649e-4340-9bdc-d2fca1296e8c service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:42.575996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c0b271c3-649e-4340-9bdc-d2fca1296e8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:42.593365 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d426e23e-5e8c-471a-973b-6af594be21a8 None None] MFA Rules not processed for user `5ef698c4f57f44d387550921a288d7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:42.605488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d426e23e-5e8c-471a-973b-6af594be21a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:42.727178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 851/1720] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:42.737293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb34502-ff37-4c42-aac8-4edbb77821e4 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:42.738158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb34502-ff37-4c42-aac8-4edbb77821e4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:42.738573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb34502-ff37-4c42-aac8-4edbb77821e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:42.781184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9eb34502-ff37-4c42-aac8-4edbb77821e4 None None] MFA Rules not processed for user `5ef698c4f57f44d387550921a288d7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:42.782788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9eb34502-ff37-4c42-aac8-4edbb77821e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:42.794512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 852/1721] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:12:42.802549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb34502-ff37-4c42-aac8-4edbb77821e4 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:42.808171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9eb34502-ff37-4c42-aac8-4edbb77821e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:42.937964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b271c3-649e-4340-9bdc-d2fca1296e8c service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:42.958352 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc68b2c-c4fd-4895-bf48-dc4e2a0dcbaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:42.959017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc68b2c-c4fd-4895-bf48-dc4e2a0dcbaa service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:42.960619 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc68b2c-c4fd-4895-bf48-dc4e2a0dcbaa service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:42.960912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc68b2c-c4fd-4895-bf48-dc4e2a0dcbaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:42.970859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3bc68b2c-c4fd-4895-bf48-dc4e2a0dcbaa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:42.985907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb34502-ff37-4c42-aac8-4edbb77821e4 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:42.999974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0deec0-e9ca-40ca-9afc-f5dec8e5435d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:43.002303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0deec0-e9ca-40ca-9afc-f5dec8e5435d service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:43.003149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0deec0-e9ca-40ca-9afc-f5dec8e5435d service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:43.004656 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0deec0-e9ca-40ca-9afc-f5dec8e5435d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:43.023786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0deec0-e9ca-40ca-9afc-f5dec8e5435d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:43.027887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0deec0-e9ca-40ca-9afc-f5dec8e5435d service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:43.070143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 853/1722] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:43.167949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc68b2c-c4fd-4895-bf48-dc4e2a0dcbaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:43.170992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc68b2c-c4fd-4895-bf48-dc4e2a0dcbaa service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:43.191169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 870/1723] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:12:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 18:12:43.420756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:43.682133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0deec0-e9ca-40ca-9afc-f5dec8e5435d service nova] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:43.684838 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fa0deec0-e9ca-40ca-9afc-f5dec8e5435d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:43.728419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0deec0-e9ca-40ca-9afc-f5dec8e5435d service nova] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:43.734585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:43.735260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:43.735553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:43.735836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] PATH_INFO: `/v3/projects/e31226e571684364a55b376c28f089d2` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:43.737098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e31226e571684364a55b376c28f089d2)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:43.739185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:43.782496 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:43.920022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e31226e571684364a55b376c28f089d2', 'request_id': 'req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:43.947561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e31226e571684364a55b376c28f089d2', 'request_id': 'req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:44.067048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e31226e571684364a55b376c28f089d2', 'request_id': 'req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:44.093283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] The token cache is being invalidate because project e31226e571684364a55b376c28f089d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:12:44.095354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:44.097336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2e0a8898-b0fd-494d-bd5c-d9068771b1fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:44.105381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 854/1724] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:12:43 2022] DELETE /identity/v3/projects/e31226e571684364a55b376c28f089d2 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:44.110612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309fe34-59fa-4962-b8fb-a63f2a6ac4f0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:44.111481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309fe34-59fa-4962-b8fb-a63f2a6ac4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:44.112200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309fe34-59fa-4962-b8fb-a63f2a6ac4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:44.150322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2309fe34-59fa-4962-b8fb-a63f2a6ac4f0 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:44.151648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2309fe34-59fa-4962-b8fb-a63f2a6ac4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:44.324566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 871/1725] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:44.336551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911bc8-2ce7-42ca-9c91-c1fb50d16794 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:44.337081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911bc8-2ce7-42ca-9c91-c1fb50d16794 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:44.337467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911bc8-2ce7-42ca-9c91-c1fb50d16794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:44.394516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3a911bc8-2ce7-42ca-9c91-c1fb50d16794 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:44.398108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3a911bc8-2ce7-42ca-9c91-c1fb50d16794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:44.444534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 855/1726] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:12:44.448748 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309fe34-59fa-4962-b8fb-a63f2a6ac4f0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:44.463263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309fe34-59fa-4962-b8fb-a63f2a6ac4f0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:44.475566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b556a0f-b0a1-4e44-a717-a64b6bc14c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:44.476910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b556a0f-b0a1-4e44-a717-a64b6bc14c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:44.477453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b556a0f-b0a1-4e44-a717-a64b6bc14c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:44.478082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b556a0f-b0a1-4e44-a717-a64b6bc14c46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:44.479223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b556a0f-b0a1-4e44-a717-a64b6bc14c46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:44.479761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b556a0f-b0a1-4e44-a717-a64b6bc14c46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:44.483970 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b556a0f-b0a1-4e44-a717-a64b6bc14c46 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:44.501562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 872/1727] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:44.506151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a911bc8-2ce7-42ca-9c91-c1fb50d16794 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:44.529228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a911bc8-2ce7-42ca-9c91-c1fb50d16794 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:44.558949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13715fc5-4700-4ddb-ad69-9f5c68d69691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:44.559878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13715fc5-4700-4ddb-ad69-9f5c68d69691 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:44.567426 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13715fc5-4700-4ddb-ad69-9f5c68d69691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:44.568372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13715fc5-4700-4ddb-ad69-9f5c68d69691 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:44.575215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13715fc5-4700-4ddb-ad69-9f5c68d69691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:44.598407 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13715fc5-4700-4ddb-ad69-9f5c68d69691 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:45.095308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 856/1728] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:44 2022] POST /identity/v3/projects => generated 389 bytes in 579 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:45.101341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b556a0f-b0a1-4e44-a717-a64b6bc14c46 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:45.102328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13715fc5-4700-4ddb-ad69-9f5c68d69691 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:45.104876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-13715fc5-4700-4ddb-ad69-9f5c68d69691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:45.140145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b556a0f-b0a1-4e44-a717-a64b6bc14c46 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:45.369043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13715fc5-4700-4ddb-ad69-9f5c68d69691 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:45.377184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:45.379987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:45.382820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:45.383334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] PATH_INFO: `/v3/projects/bb2a698a9b6a4d9a9b67a93685997f21` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:45.387748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb2a698a9b6a4d9a9b67a93685997f21)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:45.397192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:45.398888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e84d43-036c-4142-b8e4-e1378543b977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:45.407006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e84d43-036c-4142-b8e4-e1378543b977 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:45.407006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e84d43-036c-4142-b8e4-e1378543b977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:45.407006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e84d43-036c-4142-b8e4-e1378543b977 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:45.411406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e84d43-036c-4142-b8e4-e1378543b977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:45.435347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e84d43-036c-4142-b8e4-e1378543b977 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:45.625759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 873/1729] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:45 2022] POST /identity/v3/users => generated 295 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:12:45.629617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e84d43-036c-4142-b8e4-e1378543b977 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:45.678437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e84d43-036c-4142-b8e4-e1378543b977 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:45.699045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2a698a9b6a4d9a9b67a93685997f21', 'request_id': 'req-30705297-4fdd-4bf0-9a9b-3127c04da9b2'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:45.730347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2a698a9b6a4d9a9b67a93685997f21', 'request_id': 'req-30705297-4fdd-4bf0-9a9b-3127c04da9b2'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:45.789744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2a698a9b6a4d9a9b67a93685997f21', 'request_id': 'req-30705297-4fdd-4bf0-9a9b-3127c04da9b2'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:45.824740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] The token cache is being invalidate because project bb2a698a9b6a4d9a9b67a93685997f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:12:45.825153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:45.837634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:45.839502 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 857/1730] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:12:45 2022] DELETE /identity/v3/projects/bb2a698a9b6a4d9a9b67a93685997f21 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:45.844267 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:45.853893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:45.894869 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:46.006924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db01569-a6e4-4120-91d1-20e9077b33d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:46.007660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db01569-a6e4-4120-91d1-20e9077b33d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:46.008022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db01569-a6e4-4120-91d1-20e9077b33d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:46.008378 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db01569-a6e4-4120-91d1-20e9077b33d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:46.009204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db01569-a6e4-4120-91d1-20e9077b33d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:46.025166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db01569-a6e4-4120-91d1-20e9077b33d7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:46.051697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 874/1731] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:12:46.053658 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31e558-34ae-45f0-90ab-a29b75d3e92f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:46.053962 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31e558-34ae-45f0-90ab-a29b75d3e92f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:46.054332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31e558-34ae-45f0-90ab-a29b75d3e92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:46.144880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7a31e558-34ae-45f0-90ab-a29b75d3e92f None None] MFA Rules not processed for user `5ef698c4f57f44d387550921a288d7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:46.146161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7a31e558-34ae-45f0-90ab-a29b75d3e92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:46.147379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30705297-4fdd-4bf0-9a9b-3127c04da9b2 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:46.173376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:46.174253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:46.175148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:46.175515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] PATH_INFO: `/v3/users/684b7940558644d686d59f86967add68` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:46.177769 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=684b7940558644d686d59f86967add68)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:46.183079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:46.316212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 875/1732] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:46.318304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0967b90-28a2-4d87-9ddd-dcc2312af5fc None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:46.319283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0967b90-28a2-4d87-9ddd-dcc2312af5fc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:46.320242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0967b90-28a2-4d87-9ddd-dcc2312af5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:46.348302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b0967b90-28a2-4d87-9ddd-dcc2312af5fc None None] MFA Rules not processed for user `b038336582a0435dbd47ad2c09cd23e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:46.351107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b0967b90-28a2-4d87-9ddd-dcc2312af5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:46.495049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 876/1733] 10.222.0.93 () {58 vars in 1015 bytes} [Thu Jul 28 18:12:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:46.496537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0967b90-28a2-4d87-9ddd-dcc2312af5fc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:46.501045 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:46.502445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b0967b90-28a2-4d87-9ddd-dcc2312af5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:46.511245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '684b7940558644d686d59f86967add68'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:46.545248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '684b7940558644d686d59f86967add68'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:46.546113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0967b90-28a2-4d87-9ddd-dcc2312af5fc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:46.564895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eed4b-5f97-4cd6-8a5d-a12cb0630944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:46.568186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eed4b-5f97-4cd6-8a5d-a12cb0630944 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:46.569235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eed4b-5f97-4cd6-8a5d-a12cb0630944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:46.570980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eed4b-5f97-4cd6-8a5d-a12cb0630944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:46.571928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13eed4b-5f97-4cd6-8a5d-a12cb0630944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:46.580788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13eed4b-5f97-4cd6-8a5d-a12cb0630944 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:46.609281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 877/1734] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:46.610937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eed4b-5f97-4cd6-8a5d-a12cb0630944 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:46.624034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e13eed4b-5f97-4cd6-8a5d-a12cb0630944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:46.647943 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '684b7940558644d686d59f86967add68'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:46.648858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eed4b-5f97-4cd6-8a5d-a12cb0630944 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:46.649319 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '684b7940558644d686d59f86967add68'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:46.683651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d300b3-30b0-4623-8bcd-1e45b4da7adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:46.690825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d300b3-30b0-4623-8bcd-1e45b4da7adf service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:46.692179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d300b3-30b0-4623-8bcd-1e45b4da7adf service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:46.692478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d300b3-30b0-4623-8bcd-1e45b4da7adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:46.695636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '684b7940558644d686d59f86967add68'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:46.698735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f1d300b3-30b0-4623-8bcd-1e45b4da7adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:46.706214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 858/1735] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:45 2022] DELETE /identity/v3/users/684b7940558644d686d59f86967add68 => generated 0 bytes in 862 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:46.707691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:46.709451 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:46.780320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d300b3-30b0-4623-8bcd-1e45b4da7adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:46.782037 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d300b3-30b0-4623-8bcd-1e45b4da7adf service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:46.802284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 878/1736] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:46.940920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27877515-e252-44db-a187-05e0ddd0c562 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:46.957599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bf8bb-54e7-426a-8e48-d02efc3dee56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:46.958474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bf8bb-54e7-426a-8e48-d02efc3dee56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:46.958867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bf8bb-54e7-426a-8e48-d02efc3dee56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:46.959842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bf8bb-54e7-426a-8e48-d02efc3dee56 admin admin] PATH_INFO: `/v3/projects/6ba0bf8a83984c37a31afaff6295a882/users/a3b1a7f6d9224237af97e8287add9539/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:47.004807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bf8bb-54e7-426a-8e48-d02efc3dee56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ba0bf8a83984c37a31afaff6295a882, user_id=a3b1a7f6d9224237af97e8287add9539, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:47.014030 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bf8bb-54e7-426a-8e48-d02efc3dee56 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:47.136769 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 859/1737] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:46 2022] PUT /identity/v3/projects/6ba0bf8a83984c37a31afaff6295a882/users/a3b1a7f6d9224237af97e8287add9539/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:47.163759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdcaff-3fc0-4e5e-80c6-ac7f2a8d81bf None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:47.164429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdcaff-3fc0-4e5e-80c6-ac7f2a8d81bf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:47.164429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdcaff-3fc0-4e5e-80c6-ac7f2a8d81bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:47.218197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5cbdcaff-3fc0-4e5e-80c6-ac7f2a8d81bf None None] MFA Rules not processed for user `a3b1a7f6d9224237af97e8287add9539`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:47.218197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5cbdcaff-3fc0-4e5e-80c6-ac7f2a8d81bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:47.336849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 879/1738] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:47.340719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bf8bb-54e7-426a-8e48-d02efc3dee56 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:47.388496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb9b6b-4a0a-48eb-8f2a-2c2748657531 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:47.388911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb9b6b-4a0a-48eb-8f2a-2c2748657531 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:47.389254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb9b6b-4a0a-48eb-8f2a-2c2748657531 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:47.391140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 880/1739] 10.222.0.86 () {54 vars in 912 bytes} [Thu Jul 28 18:12:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:12:47.402284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b53ce-4d28-4d4b-ad71-8a8ac818b239 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:47.407757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b53ce-4d28-4d4b-ad71-8a8ac818b239 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:47.408262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b53ce-4d28-4d4b-ad71-8a8ac818b239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:47.451945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bf8bb-54e7-426a-8e48-d02efc3dee56 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:47.466523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1ad5b-5c51-4b73-af25-042899e67605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:47.468605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1ad5b-5c51-4b73-af25-042899e67605 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:47.468605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1ad5b-5c51-4b73-af25-042899e67605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:47.469709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1ad5b-5c51-4b73-af25-042899e67605 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:47.470990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1ad5b-5c51-4b73-af25-042899e67605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:47.476801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1ad5b-5c51-4b73-af25-042899e67605 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:47.477438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-151b53ce-4d28-4d4b-ad71-8a8ac818b239 None None] MFA Rules not processed for user `b038336582a0435dbd47ad2c09cd23e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:47.478812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-151b53ce-4d28-4d4b-ad71-8a8ac818b239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:47.589339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 881/1740] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:12:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:47.645024 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b53ce-4d28-4d4b-ad71-8a8ac818b239 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:47.646904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-151b53ce-4d28-4d4b-ad71-8a8ac818b239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:47.663124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b53ce-4d28-4d4b-ad71-8a8ac818b239 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:47.671921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 860/1741] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:47 2022] POST /identity/v3/projects => generated 387 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:47.672142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:47.681922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1ad5b-5c51-4b73-af25-042899e67605 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:47.762674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf002cc-72bb-4f2a-888f-fcf71127556a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:47.763692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf002cc-72bb-4f2a-888f-fcf71127556a service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:47.764801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf002cc-72bb-4f2a-888f-fcf71127556a service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:47.765345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf002cc-72bb-4f2a-888f-fcf71127556a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:47.781298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf002cc-72bb-4f2a-888f-fcf71127556a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:47.785612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf002cc-72bb-4f2a-888f-fcf71127556a service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:47.828161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 882/1742] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:47.872831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1ad5b-5c51-4b73-af25-042899e67605 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:47.879904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ee68d-b9e9-47af-b2d2-fc1fd54cc15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:47.880933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ee68d-b9e9-47af-b2d2-fc1fd54cc15e admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:47.881329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ee68d-b9e9-47af-b2d2-fc1fd54cc15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:47.881859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ee68d-b9e9-47af-b2d2-fc1fd54cc15e admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:47.882774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ee68d-b9e9-47af-b2d2-fc1fd54cc15e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:47.886009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ee68d-b9e9-47af-b2d2-fc1fd54cc15e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:48.029267 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 861/1743] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:47 2022] POST /identity/v3/users => generated 300 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:48.033785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf002cc-72bb-4f2a-888f-fcf71127556a service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:48.098894 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf002cc-72bb-4f2a-888f-fcf71127556a service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:48.115168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104d630-ecb7-4ad2-a827-a3abbc0be8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:48.119334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104d630-ecb7-4ad2-a827-a3abbc0be8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:48.121745 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104d630-ecb7-4ad2-a827-a3abbc0be8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:48.122506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104d630-ecb7-4ad2-a827-a3abbc0be8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:48.125084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104d630-ecb7-4ad2-a827-a3abbc0be8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:48.142920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104d630-ecb7-4ad2-a827-a3abbc0be8d4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:48.170689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 883/1744] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:48.178383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ee68d-b9e9-47af-b2d2-fc1fd54cc15e admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:48.229927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ee68d-b9e9-47af-b2d2-fc1fd54cc15e admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:48.256423 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2202dd-7fd4-4bf7-9995-d1ad3e70537d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:48.263422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2202dd-7fd4-4bf7-9995-d1ad3e70537d admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:48.268115 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2202dd-7fd4-4bf7-9995-d1ad3e70537d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:48.268575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2202dd-7fd4-4bf7-9995-d1ad3e70537d admin admin] PATH_INFO: `/v3/projects/447dce0ebb3c42e183b05c12e8c3dd33/users/64da8653cc2a41e38b3e748849120367/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:48.324321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104d630-ecb7-4ad2-a827-a3abbc0be8d4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:48.328618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e104d630-ecb7-4ad2-a827-a3abbc0be8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:48.330987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2202dd-7fd4-4bf7-9995-d1ad3e70537d admin admin] RBAC: Authorizing `identity:create_grant(project_id=447dce0ebb3c42e183b05c12e8c3dd33, user_id=64da8653cc2a41e38b3e748849120367, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:48.342936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2202dd-7fd4-4bf7-9995-d1ad3e70537d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:48.355704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104d630-ecb7-4ad2-a827-a3abbc0be8d4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:48.363552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:48.364686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:48.366764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:48.368250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] PATH_INFO: `/v3/projects/d80959bc029e4d539a67e10637702a86` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:48.371059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d80959bc029e4d539a67e10637702a86)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:48.372541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 18:12:48.372541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 28 18:12:48.372541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: Jul 28 18:12:48.375864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:48.475652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 862/1745] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:48 2022] PUT /identity/v3/projects/447dce0ebb3c42e183b05c12e8c3dd33/users/64da8653cc2a41e38b3e748849120367/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:48.491006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2202dd-7fd4-4bf7-9995-d1ad3e70537d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:48.492100 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:48.551145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd80959bc029e4d539a67e10637702a86', 'request_id': 'req-a878d603-7892-4262-a61e-63b7f81a1e5b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:48.557262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd80959bc029e4d539a67e10637702a86', 'request_id': 'req-a878d603-7892-4262-a61e-63b7f81a1e5b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:48.577521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd80959bc029e4d539a67e10637702a86', 'request_id': 'req-a878d603-7892-4262-a61e-63b7f81a1e5b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:48.614495 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2202dd-7fd4-4bf7-9995-d1ad3e70537d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:48.649264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] The token cache is being invalidate because project d80959bc029e4d539a67e10637702a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:12:48.649953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:48.651003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a878d603-7892-4262-a61e-63b7f81a1e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:48.654222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 884/1746] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:12:48 2022] DELETE /identity/v3/projects/d80959bc029e4d539a67e10637702a86 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:48.656204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e1c4-3d2e-494d-aa07-61f5f46cca94 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:48.656598 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e1c4-3d2e-494d-aa07-61f5f46cca94 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:48.656996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e1c4-3d2e-494d-aa07-61f5f46cca94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:48.688274 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-68e4e1c4-3d2e-494d-aa07-61f5f46cca94 None None] MFA Rules not processed for user `b9874df18af245688db1137b6f1b0bff`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:48.689918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-68e4e1c4-3d2e-494d-aa07-61f5f46cca94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:48.703875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997e95f-58f1-4638-bf42-890d73a5bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:48.704534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997e95f-58f1-4638-bf42-890d73a5bd29 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:48.704827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997e95f-58f1-4638-bf42-890d73a5bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:48.705146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997e95f-58f1-4638-bf42-890d73a5bd29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:48.705881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997e95f-58f1-4638-bf42-890d73a5bd29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:48.707719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997e95f-58f1-4638-bf42-890d73a5bd29 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:48.720353 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 863/1747] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:48.726011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5037d-d6ce-45c9-8305-61414721b3c2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:48.726011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5037d-d6ce-45c9-8305-61414721b3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:48.726011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5037d-d6ce-45c9-8305-61414721b3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:48.775268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c6c5037d-d6ce-45c9-8305-61414721b3c2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:48.778631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c6c5037d-d6ce-45c9-8305-61414721b3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:48.796247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 885/1748] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:48.797633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4e1c4-3d2e-494d-aa07-61f5f46cca94 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:48.800720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 864/1749] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:48.808903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee688f7e-72ff-4a84-bcb0-e50385d763a6 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:48.809221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee688f7e-72ff-4a84-bcb0-e50385d763a6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:48.809502 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee688f7e-72ff-4a84-bcb0-e50385d763a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:48.810181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-68e4e1c4-3d2e-494d-aa07-61f5f46cca94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:48.829878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4e1c4-3d2e-494d-aa07-61f5f46cca94 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:48.836034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b7c5d-02c5-4b82-a974-09e8feef8f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:48.836923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b7c5d-02c5-4b82-a974-09e8feef8f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:48.837240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b7c5d-02c5-4b82-a974-09e8feef8f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:48.837582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b7c5d-02c5-4b82-a974-09e8feef8f92 admin admin] PATH_INFO: `/v3/projects/447dce0ebb3c42e183b05c12e8c3dd33/users/64da8653cc2a41e38b3e748849120367/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:48.838338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-ee688f7e-72ff-4a84-bcb0-e50385d763a6 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:48.842687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b7c5d-02c5-4b82-a974-09e8feef8f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=447dce0ebb3c42e183b05c12e8c3dd33, user_id=64da8653cc2a41e38b3e748849120367, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:48.847517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b7c5d-02c5-4b82-a974-09e8feef8f92 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:48.847811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ee688f7e-72ff-4a84-bcb0-e50385d763a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:48.862261 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 886/1750] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:48 2022] PUT /identity/v3/projects/447dce0ebb3c42e183b05c12e8c3dd33/users/64da8653cc2a41e38b3e748849120367/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:48.871341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:48.873929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb18a14-36b4-481b-aac9-61a5085deb2e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:48.875254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb18a14-36b4-481b-aac9-61a5085deb2e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:48.875751 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb18a14-36b4-481b-aac9-61a5085deb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:48.965576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1eb18a14-36b4-481b-aac9-61a5085deb2e None None] MFA Rules not processed for user `64da8653cc2a41e38b3e748849120367`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:48.971682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1eb18a14-36b4-481b-aac9-61a5085deb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:49.000298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 865/1751] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:12:49.004372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee688f7e-72ff-4a84-bcb0-e50385d763a6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:49.019465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee688f7e-72ff-4a84-bcb0-e50385d763a6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:49.032673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55d616-984d-418a-af3e-fa5e7b9a718d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:49.037751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55d616-984d-418a-af3e-fa5e7b9a718d admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:49.044739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55d616-984d-418a-af3e-fa5e7b9a718d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:49.044739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55d616-984d-418a-af3e-fa5e7b9a718d admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:49.044739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55d616-984d-418a-af3e-fa5e7b9a718d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:49.044739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55d616-984d-418a-af3e-fa5e7b9a718d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:49.044739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55d616-984d-418a-af3e-fa5e7b9a718d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:49.078345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 866/1752] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:49.079276 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55d616-984d-418a-af3e-fa5e7b9a718d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:49.094761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55d616-984d-418a-af3e-fa5e7b9a718d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:49.109083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7d40c-bd0c-42fd-80c3-f7fd5d15fb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:49.109388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7d40c-bd0c-42fd-80c3-f7fd5d15fb11 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:49.109681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7d40c-bd0c-42fd-80c3-f7fd5d15fb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:49.109935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7d40c-bd0c-42fd-80c3-f7fd5d15fb11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:49.110614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7d40c-bd0c-42fd-80c3-f7fd5d15fb11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:49.114658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7d40c-bd0c-42fd-80c3-f7fd5d15fb11 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:49.176807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 887/1753] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:49.202719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb18a14-36b4-481b-aac9-61a5085deb2e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:49.204552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1eb18a14-36b4-481b-aac9-61a5085deb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:49.239903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 867/1754] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:49 2022] POST /identity/v3/projects => generated 369 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:49.244017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7d40c-bd0c-42fd-80c3-f7fd5d15fb11 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:49.312794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb18a14-36b4-481b-aac9-61a5085deb2e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:49.373797 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7d40c-bd0c-42fd-80c3-f7fd5d15fb11 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:49.384099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6612e-3d7b-4f7f-adbb-e1d857c3795d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:49.384883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6612e-3d7b-4f7f-adbb-e1d857c3795d admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:49.385323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6612e-3d7b-4f7f-adbb-e1d857c3795d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:49.385679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6612e-3d7b-4f7f-adbb-e1d857c3795d admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:49.386491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6612e-3d7b-4f7f-adbb-e1d857c3795d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:49.390780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6612e-3d7b-4f7f-adbb-e1d857c3795d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:49.443931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfdad0-8289-41bc-9add-d17e80b3c945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:49.446716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfdad0-8289-41bc-9add-d17e80b3c945 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:49.446716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfdad0-8289-41bc-9add-d17e80b3c945 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:49.446716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfdad0-8289-41bc-9add-d17e80b3c945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:49.505101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 868/1755] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:49 2022] POST /identity/v3/users => generated 285 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:49.506528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6612e-3d7b-4f7f-adbb-e1d857c3795d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:49.521303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6612e-3d7b-4f7f-adbb-e1d857c3795d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:49.539178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ddbce-c889-49fd-88f9-ccee93e16dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:49.540569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ddbce-c889-49fd-88f9-ccee93e16dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:49.541088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ddbce-c889-49fd-88f9-ccee93e16dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:49.541555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ddbce-c889-49fd-88f9-ccee93e16dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:49.542454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ddbce-c889-49fd-88f9-ccee93e16dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:49.544298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ddbce-c889-49fd-88f9-ccee93e16dd2 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:49.568945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 869/1756] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:49.569549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ddbce-c889-49fd-88f9-ccee93e16dd2 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:49.596311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfdad0-8289-41bc-9add-d17e80b3c945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:49.597989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ddbce-c889-49fd-88f9-ccee93e16dd2 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:49.600224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfdad0-8289-41bc-9add-d17e80b3c945 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:49.615582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc20c39-4195-43c6-9e29-97fd28703977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:49.616426 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc20c39-4195-43c6-9e29-97fd28703977 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:49.616743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc20c39-4195-43c6-9e29-97fd28703977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:49.617062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc20c39-4195-43c6-9e29-97fd28703977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:49.625611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc20c39-4195-43c6-9e29-97fd28703977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:49.627557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc20c39-4195-43c6-9e29-97fd28703977 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:49.630690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 888/1757] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:49.673489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 870/1758] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:49.677928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfdad0-8289-41bc-9add-d17e80b3c945 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:49.692754 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfdad0-8289-41bc-9add-d17e80b3c945 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:49.698272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b60360-2d25-4ba7-b9ef-43142863cb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:49.699392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b60360-2d25-4ba7-b9ef-43142863cb93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:49.699778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b60360-2d25-4ba7-b9ef-43142863cb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:49.700129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b60360-2d25-4ba7-b9ef-43142863cb93 admin admin] PATH_INFO: `/v3/projects/bf3142a6a4524ff1986f13f92c1c6cb5/users/4fa961aecf2d45ec98992f88c24857a2/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:49.735466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:49.770908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b60360-2d25-4ba7-b9ef-43142863cb93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf3142a6a4524ff1986f13f92c1c6cb5, user_id=4fa961aecf2d45ec98992f88c24857a2, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:49.774017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b60360-2d25-4ba7-b9ef-43142863cb93 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:49.892333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 889/1759] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:49 2022] PUT /identity/v3/projects/bf3142a6a4524ff1986f13f92c1c6cb5/users/4fa961aecf2d45ec98992f88c24857a2/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:49.893388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc20c39-4195-43c6-9e29-97fd28703977 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:49.895652 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84c696-f0e4-4297-be4b-4d7929dfead7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:49.895902 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84c696-f0e4-4297-be4b-4d7929dfead7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:49.896187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84c696-f0e4-4297-be4b-4d7929dfead7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:49.904713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc20c39-4195-43c6-9e29-97fd28703977 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:49.953364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-cc84c696-f0e4-4297-be4b-4d7929dfead7 None None] MFA Rules not processed for user `4fa961aecf2d45ec98992f88c24857a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:49.955758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cc84c696-f0e4-4297-be4b-4d7929dfead7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:49.991459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413fd16-a3eb-4b73-8131-d98eb659406e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:49.992394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413fd16-a3eb-4b73-8131-d98eb659406e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:49.992832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413fd16-a3eb-4b73-8131-d98eb659406e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:49.993301 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413fd16-a3eb-4b73-8131-d98eb659406e admin admin] PATH_INFO: `/v3/users/a3b1a7f6d9224237af97e8287add9539` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:49.997107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413fd16-a3eb-4b73-8131-d98eb659406e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b1a7f6d9224237af97e8287add9539)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:50.001253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413fd16-a3eb-4b73-8131-d98eb659406e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:50.007592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:50.089225 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 890/1760] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:12:50.090564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84c696-f0e4-4297-be4b-4d7929dfead7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:50.105366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84c696-f0e4-4297-be4b-4d7929dfead7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:50.113299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37801e8b-31ac-44d5-a3dc-ab0145efdc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:50.114108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37801e8b-31ac-44d5-a3dc-ab0145efdc97 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:50.114513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37801e8b-31ac-44d5-a3dc-ab0145efdc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:50.114891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37801e8b-31ac-44d5-a3dc-ab0145efdc97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:50.115933 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37801e8b-31ac-44d5-a3dc-ab0145efdc97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:50.118162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37801e8b-31ac-44d5-a3dc-ab0145efdc97 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:50.218105 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 891/1761] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:50 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:50.220030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37801e8b-31ac-44d5-a3dc-ab0145efdc97 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:50.316363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-4413fd16-a3eb-4b73-8131-d98eb659406e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b1a7f6d9224237af97e8287add9539'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:50.346061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-4413fd16-a3eb-4b73-8131-d98eb659406e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b1a7f6d9224237af97e8287add9539'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:50.386530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37801e8b-31ac-44d5-a3dc-ab0145efdc97 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:50.396255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fc820-81f3-4e1e-b98e-f69aea5a8489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:50.396996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fc820-81f3-4e1e-b98e-f69aea5a8489 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:50.397399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fc820-81f3-4e1e-b98e-f69aea5a8489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:50.397772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fc820-81f3-4e1e-b98e-f69aea5a8489 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:50.399747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fc820-81f3-4e1e-b98e-f69aea5a8489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:50.410156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fc820-81f3-4e1e-b98e-f69aea5a8489 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:50.446376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-4413fd16-a3eb-4b73-8131-d98eb659406e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b1a7f6d9224237af97e8287add9539'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:50.447916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-4413fd16-a3eb-4b73-8131-d98eb659406e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b1a7f6d9224237af97e8287add9539'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:50.472291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-4413fd16-a3eb-4b73-8131-d98eb659406e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b1a7f6d9224237af97e8287add9539'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:50.493437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 871/1762] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:49 2022] DELETE /identity/v3/users/a3b1a7f6d9224237af97e8287add9539 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:50.498242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93dae0-3067-493e-a77d-e45f1423abcc None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:50.499013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93dae0-3067-493e-a77d-e45f1423abcc None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:50.499294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93dae0-3067-493e-a77d-e45f1423abcc None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:50.501192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 872/1763] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:12:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:12:50.502383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72419ed-dcf5-4c0f-aa99-fd911724ed82 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:50.502678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72419ed-dcf5-4c0f-aa99-fd911724ed82 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:50.502957 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72419ed-dcf5-4c0f-aa99-fd911724ed82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:50.538923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e72419ed-dcf5-4c0f-aa99-fd911724ed82 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:50.540428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e72419ed-dcf5-4c0f-aa99-fd911724ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:50.580599 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:50.644903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 892/1764] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:50 2022] POST /identity/v3/users => generated 291 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:50.646667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d505c5-4bc5-4822-8f86-b4a02b00123a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:50.646964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d505c5-4bc5-4822-8f86-b4a02b00123a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:50.647246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d505c5-4bc5-4822-8f86-b4a02b00123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:50.688917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d5d505c5-4bc5-4822-8f86-b4a02b00123a None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:50.694859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 873/1765] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:50.694859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72419ed-dcf5-4c0f-aa99-fd911724ed82 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:50.695147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d5d505c5-4bc5-4822-8f86-b4a02b00123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:50.696799 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e72419ed-dcf5-4c0f-aa99-fd911724ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:50.722030 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72419ed-dcf5-4c0f-aa99-fd911724ed82 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:50.740900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2606007-4c0a-43ee-87a8-767167231efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:50.741502 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2606007-4c0a-43ee-87a8-767167231efe admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:50.741788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2606007-4c0a-43ee-87a8-767167231efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:50.742072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2606007-4c0a-43ee-87a8-767167231efe admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:50.742811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2606007-4c0a-43ee-87a8-767167231efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:50.745806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2606007-4c0a-43ee-87a8-767167231efe admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:50.772341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 874/1766] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:50.780995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2606007-4c0a-43ee-87a8-767167231efe admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:50.784102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b2606007-4c0a-43ee-87a8-767167231efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:50.914101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 893/1767] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:12:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:50.915604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d505c5-4bc5-4822-8f86-b4a02b00123a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:50.945614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d505c5-4bc5-4822-8f86-b4a02b00123a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:50.954152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2606007-4c0a-43ee-87a8-767167231efe admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:50.962925 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f6bf8-ff9c-426b-86a4-03e041612040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:50.963629 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f6bf8-ff9c-426b-86a4-03e041612040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:50.963910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f6bf8-ff9c-426b-86a4-03e041612040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:50.964186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f6bf8-ff9c-426b-86a4-03e041612040 admin admin] PATH_INFO: `/v3/projects/d25fdc3d0497443aa8a4762d03f8abcd/users/6432b726eb34478cbf8d57e2acaea12f/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:50.969475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f417ba-da0a-423c-b3ac-58231311b611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:50.970299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f417ba-da0a-423c-b3ac-58231311b611 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:50.970586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f417ba-da0a-423c-b3ac-58231311b611 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:50.970867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f417ba-da0a-423c-b3ac-58231311b611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:50.994639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f417ba-da0a-423c-b3ac-58231311b611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:50.999405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f417ba-da0a-423c-b3ac-58231311b611 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:51.006731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f6bf8-ff9c-426b-86a4-03e041612040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25fdc3d0497443aa8a4762d03f8abcd, user_id=6432b726eb34478cbf8d57e2acaea12f, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:51.009623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f6bf8-ff9c-426b-86a4-03e041612040 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:51.021511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 875/1768] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:51.022459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f417ba-da0a-423c-b3ac-58231311b611 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:51.024155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d5f417ba-da0a-423c-b3ac-58231311b611 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:51.045035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f417ba-da0a-423c-b3ac-58231311b611 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:51.054875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba87d5-137a-411b-a4cc-adaaef309fc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:51.055573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba87d5-137a-411b-a4cc-adaaef309fc0 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:51.055864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba87d5-137a-411b-a4cc-adaaef309fc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:51.056139 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba87d5-137a-411b-a4cc-adaaef309fc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:51.067345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba87d5-137a-411b-a4cc-adaaef309fc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:51.070345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba87d5-137a-411b-a4cc-adaaef309fc0 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:51.083502 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 876/1769] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:51.085024 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed66a-86d6-43a6-a25a-a3cbfc514822 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:51.085320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed66a-86d6-43a6-a25a-a3cbfc514822 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:51.085735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ed66a-86d6-43a6-a25a-a3cbfc514822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:51.120871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 894/1770] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:50 2022] PUT /identity/v3/projects/d25fdc3d0497443aa8a4762d03f8abcd/users/6432b726eb34478cbf8d57e2acaea12f/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:51.122056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f6bf8-ff9c-426b-86a4-03e041612040 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:51.125384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a90ed66a-86d6-43a6-a25a-a3cbfc514822 None None] MFA Rules not processed for user `ca7d773290a9483d91ebefad341addee`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:51.127212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a90ed66a-86d6-43a6-a25a-a3cbfc514822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:51.270695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 877/1771] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:51.270902 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f6bf8-ff9c-426b-86a4-03e041612040 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:51.275774 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f11fc-fc6c-46bf-a8cf-453412c54185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:51.276374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f11fc-fc6c-46bf-a8cf-453412c54185 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:51.276657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f11fc-fc6c-46bf-a8cf-453412c54185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:51.276934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f11fc-fc6c-46bf-a8cf-453412c54185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:51.277877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f11fc-fc6c-46bf-a8cf-453412c54185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:51.279487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f11fc-fc6c-46bf-a8cf-453412c54185 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:51.290976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 895/1772] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:51.295882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ed66a-86d6-43a6-a25a-a3cbfc514822 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:51.307882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ed66a-86d6-43a6-a25a-a3cbfc514822 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:51.314564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3b59d-653b-47a6-8fb0-d0e91b36acdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:51.315264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3b59d-653b-47a6-8fb0-d0e91b36acdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:51.315601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3b59d-653b-47a6-8fb0-d0e91b36acdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:51.316365 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3b59d-653b-47a6-8fb0-d0e91b36acdb admin admin] PATH_INFO: `/v3/projects/d25fdc3d0497443aa8a4762d03f8abcd/users/6432b726eb34478cbf8d57e2acaea12f/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:51.318119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3b59d-653b-47a6-8fb0-d0e91b36acdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25fdc3d0497443aa8a4762d03f8abcd, user_id=6432b726eb34478cbf8d57e2acaea12f, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:51.322348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3b59d-653b-47a6-8fb0-d0e91b36acdb admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:51.368555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 878/1773] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:51 2022] PUT /identity/v3/projects/d25fdc3d0497443aa8a4762d03f8abcd/users/6432b726eb34478cbf8d57e2acaea12f/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:51.371136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803861-f70b-4248-830f-9ccd28081c89 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:51.371576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803861-f70b-4248-830f-9ccd28081c89 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:51.371873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803861-f70b-4248-830f-9ccd28081c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:51.443323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-85803861-f70b-4248-830f-9ccd28081c89 None None] MFA Rules not processed for user `6432b726eb34478cbf8d57e2acaea12f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:51.444845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-85803861-f70b-4248-830f-9ccd28081c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:51.601359 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 896/1774] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:51 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:51.610740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3b59d-653b-47a6-8fb0-d0e91b36acdb admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:51.615555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-40e3b59d-653b-47a6-8fb0-d0e91b36acdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:51.768101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3b59d-653b-47a6-8fb0-d0e91b36acdb admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:51.781427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ab92c-a6cc-4df1-8d5c-a0131bbe195e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:51.782785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ab92c-a6cc-4df1-8d5c-a0131bbe195e service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:51.783297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ab92c-a6cc-4df1-8d5c-a0131bbe195e service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:51.783744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ab92c-a6cc-4df1-8d5c-a0131bbe195e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:51.798980 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ab92c-a6cc-4df1-8d5c-a0131bbe195e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:51.801916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ab92c-a6cc-4df1-8d5c-a0131bbe195e service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:51.830895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 879/1775] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:51 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:52.011603 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85803861-f70b-4248-830f-9ccd28081c89 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:52.017304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-85803861-f70b-4248-830f-9ccd28081c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:52.070570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85803861-f70b-4248-830f-9ccd28081c89 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:52.096735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad597b-a415-4b22-b6ba-f3324af8e32c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:52.100107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad597b-a415-4b22-b6ba-f3324af8e32c service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:52.100107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad597b-a415-4b22-b6ba-f3324af8e32c service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:52.100107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad597b-a415-4b22-b6ba-f3324af8e32c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:52.116524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ab92c-a6cc-4df1-8d5c-a0131bbe195e service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:52.118188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e21ab92c-a6cc-4df1-8d5c-a0131bbe195e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:52.123333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-76ad597b-a415-4b22-b6ba-f3324af8e32c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:52.225427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:52.383343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad597b-a415-4b22-b6ba-f3324af8e32c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:52.384919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad597b-a415-4b22-b6ba-f3324af8e32c service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:52.422395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 897/1776] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:52.446387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ab92c-a6cc-4df1-8d5c-a0131bbe195e service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:52.481654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:52.482270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:52.482592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:52.485456 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] PATH_INFO: `/v3/projects/6ba0bf8a83984c37a31afaff6295a882` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:52.493286 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ba0bf8a83984c37a31afaff6295a882)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:52.497709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:52.647443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:52.776062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ba0bf8a83984c37a31afaff6295a882', 'request_id': 'req-d27b493f-e82c-43d7-9925-f30ea74facc7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:52.794898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ba0bf8a83984c37a31afaff6295a882', 'request_id': 'req-d27b493f-e82c-43d7-9925-f30ea74facc7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:52.887834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ba0bf8a83984c37a31afaff6295a882', 'request_id': 'req-d27b493f-e82c-43d7-9925-f30ea74facc7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:52.907872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] The token cache is being invalidate because project 6ba0bf8a83984c37a31afaff6295a882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:12:52.908183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:52.909167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:52.914092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 880/1777] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:12:52 2022] DELETE /identity/v3/projects/6ba0bf8a83984c37a31afaff6295a882 => generated 0 bytes in 795 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:12:52.915933 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad597b-a415-4b22-b6ba-f3324af8e32c service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:52.952562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad597b-a415-4b22-b6ba-f3324af8e32c service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:53.055518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53572982-137c-4307-8ca3-81137cffea32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:53.056508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53572982-137c-4307-8ca3-81137cffea32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:53.056835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53572982-137c-4307-8ca3-81137cffea32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:53.057221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53572982-137c-4307-8ca3-81137cffea32 admin admin] PATH_INFO: `/v3/users/64da8653cc2a41e38b3e748849120367` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:53.073222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53572982-137c-4307-8ca3-81137cffea32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64da8653cc2a41e38b3e748849120367)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:53.075431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53572982-137c-4307-8ca3-81137cffea32 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:53.346179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-53572982-137c-4307-8ca3-81137cffea32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64da8653cc2a41e38b3e748849120367'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:53.347002 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-53572982-137c-4307-8ca3-81137cffea32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64da8653cc2a41e38b3e748849120367'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:53.377231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-53572982-137c-4307-8ca3-81137cffea32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64da8653cc2a41e38b3e748849120367'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:53.488061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-53572982-137c-4307-8ca3-81137cffea32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64da8653cc2a41e38b3e748849120367'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:53.488761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-53572982-137c-4307-8ca3-81137cffea32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64da8653cc2a41e38b3e748849120367'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:53.523019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 898/1778] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:52 2022] DELETE /identity/v3/users/64da8653cc2a41e38b3e748849120367 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:54.527863 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:54.530817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:54.555387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b493f-e82c-43d7-9925-f30ea74facc7 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:54.648264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:54.771616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:54.773625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:54.774016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:54.776011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] PATH_INFO: `/v3/projects/447dce0ebb3c42e183b05c12e8c3dd33` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:54.777595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] RBAC: Authorizing `identity:delete_project(project_id=447dce0ebb3c42e183b05c12e8c3dd33)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:54.784704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:55.023815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '447dce0ebb3c42e183b05c12e8c3dd33', 'request_id': 'req-131dd236-fe4d-4eda-bef6-8467f57c710c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:55.037652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '447dce0ebb3c42e183b05c12e8c3dd33', 'request_id': 'req-131dd236-fe4d-4eda-bef6-8467f57c710c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:55.066804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:55.213705 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '447dce0ebb3c42e183b05c12e8c3dd33', 'request_id': 'req-131dd236-fe4d-4eda-bef6-8467f57c710c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:55.234477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] The token cache is being invalidate because project 447dce0ebb3c42e183b05c12e8c3dd33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:12:55.234950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:55.237222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-131dd236-fe4d-4eda-bef6-8467f57c710c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:55.249844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b39e1-900b-4f41-9492-3612d5d1abee None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:55.251055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b39e1-900b-4f41-9492-3612d5d1abee None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:55.251055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b39e1-900b-4f41-9492-3612d5d1abee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:55.253984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 881/1779] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:12:54 2022] DELETE /identity/v3/projects/447dce0ebb3c42e183b05c12e8c3dd33 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:55.289004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c56b39e1-900b-4f41-9492-3612d5d1abee None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:55.293635 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c56b39e1-900b-4f41-9492-3612d5d1abee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:55.489852 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ab250-72b9-4345-a1f5-283ebde63d47 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:55.490139 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ab250-72b9-4345-a1f5-283ebde63d47 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:55.490418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ab250-72b9-4345-a1f5-283ebde63d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:55.495076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 899/1780] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:55.531146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-843ab250-72b9-4345-a1f5-283ebde63d47 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:55.531989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-843ab250-72b9-4345-a1f5-283ebde63d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:55.542077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 882/1781] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:55.559342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b39e1-900b-4f41-9492-3612d5d1abee None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:55.577695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b39e1-900b-4f41-9492-3612d5d1abee None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:55.589837 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ff375-cd48-48b3-8d16-0f8d16d7e1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:55.591064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ff375-cd48-48b3-8d16-0f8d16d7e1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:55.592455 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ff375-cd48-48b3-8d16-0f8d16d7e1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:55.592994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ff375-cd48-48b3-8d16-0f8d16d7e1f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:55.594501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ff375-cd48-48b3-8d16-0f8d16d7e1f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:12:55.595217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ff375-cd48-48b3-8d16-0f8d16d7e1f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:55.597256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ff375-cd48-48b3-8d16-0f8d16d7e1f7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:55.624724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 900/1782] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:12:55.626337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ab250-72b9-4345-a1f5-283ebde63d47 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:55.641773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ab250-72b9-4345-a1f5-283ebde63d47 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:55.649378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2323483-1f44-4327-b83a-138a82d0cab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:55.651230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2323483-1f44-4327-b83a-138a82d0cab4 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:55.651839 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2323483-1f44-4327-b83a-138a82d0cab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:55.652561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2323483-1f44-4327-b83a-138a82d0cab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:55.654047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2323483-1f44-4327-b83a-138a82d0cab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:55.657868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2323483-1f44-4327-b83a-138a82d0cab4 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:55.769071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 883/1783] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:12:55 2022] POST /identity/v3/projects => generated 361 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:55.773528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ff375-cd48-48b3-8d16-0f8d16d7e1f7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:55.796404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ff375-cd48-48b3-8d16-0f8d16d7e1f7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:55.971892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93b564-3c75-4398-824c-c167fad0dd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:55.972571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93b564-3c75-4398-824c-c167fad0dd91 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:55.972852 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93b564-3c75-4398-824c-c167fad0dd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:55.973191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93b564-3c75-4398-824c-c167fad0dd91 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:55.974578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93b564-3c75-4398-824c-c167fad0dd91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:55.976736 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93b564-3c75-4398-824c-c167fad0dd91 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:56.083670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 901/1784] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:12:55 2022] POST /identity/v3/users => generated 281 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:12:56.089900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2323483-1f44-4327-b83a-138a82d0cab4 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:56.148027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2323483-1f44-4327-b83a-138a82d0cab4 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:56.169755 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61373b81-7213-4102-a9de-6678664161a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:56.171761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61373b81-7213-4102-a9de-6678664161a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:56.175006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61373b81-7213-4102-a9de-6678664161a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:56.176106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61373b81-7213-4102-a9de-6678664161a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:56.194648 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61373b81-7213-4102-a9de-6678664161a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:56.204528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61373b81-7213-4102-a9de-6678664161a9 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:56.262614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 884/1785] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:56.267908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93b564-3c75-4398-824c-c167fad0dd91 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:56.298698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93b564-3c75-4398-824c-c167fad0dd91 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:56.326693 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a92da-cfae-461d-8fd7-549c7718bc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:56.329506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a92da-cfae-461d-8fd7-549c7718bc28 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:56.331028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a92da-cfae-461d-8fd7-549c7718bc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:56.333180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a92da-cfae-461d-8fd7-549c7718bc28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:56.334975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a92da-cfae-461d-8fd7-549c7718bc28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:56.338453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a92da-cfae-461d-8fd7-549c7718bc28 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:56.411846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61373b81-7213-4102-a9de-6678664161a9 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:56.414746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 902/1786] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:12:56.466896 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61373b81-7213-4102-a9de-6678664161a9 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:56.489454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1db79-2e8a-4040-a636-de237e73c752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:56.492869 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1db79-2e8a-4040-a636-de237e73c752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:56.494503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1db79-2e8a-4040-a636-de237e73c752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:56.495098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1db79-2e8a-4040-a636-de237e73c752 admin admin] PATH_INFO: `/v3/projects/25a6cfb1dbc54355813127c34dc09380/users/806d6847d3fa4fcfb422601e8b285901/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:56.547460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1db79-2e8a-4040-a636-de237e73c752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a6cfb1dbc54355813127c34dc09380, user_id=806d6847d3fa4fcfb422601e8b285901, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:56.555099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1db79-2e8a-4040-a636-de237e73c752 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:56.562039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:56.678800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 885/1787] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:12:56 2022] PUT /identity/v3/projects/25a6cfb1dbc54355813127c34dc09380/users/806d6847d3fa4fcfb422601e8b285901/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:56.693208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eca5e-5a58-4695-b2e2-700239cac669 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:56.693582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eca5e-5a58-4695-b2e2-700239cac669 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:56.693882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eca5e-5a58-4695-b2e2-700239cac669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:56.821571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-642eca5e-5a58-4695-b2e2-700239cac669 None None] MFA Rules not processed for user `806d6847d3fa4fcfb422601e8b285901`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:56.842011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-642eca5e-5a58-4695-b2e2-700239cac669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:57.025829 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 903/1788] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:57.082093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1db79-2e8a-4040-a636-de237e73c752 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:57.085094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7de1db79-2e8a-4040-a636-de237e73c752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:57.220731 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1db79-2e8a-4040-a636-de237e73c752 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:57.229508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fce54-737b-42c9-90cc-98057f23ac7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:57.230209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fce54-737b-42c9-90cc-98057f23ac7a service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:57.230596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fce54-737b-42c9-90cc-98057f23ac7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:57.231000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fce54-737b-42c9-90cc-98057f23ac7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:57.245424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fce54-737b-42c9-90cc-98057f23ac7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:57.246933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fce54-737b-42c9-90cc-98057f23ac7a service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:57.267531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 886/1789] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:57.285635 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642eca5e-5a58-4695-b2e2-700239cac669 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:57.397154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642eca5e-5a58-4695-b2e2-700239cac669 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:57.407570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e9c33-ce3b-4bb3-aade-2298095620f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:57.408846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e9c33-ce3b-4bb3-aade-2298095620f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:57.409696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e9c33-ce3b-4bb3-aade-2298095620f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:57.410216 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e9c33-ce3b-4bb3-aade-2298095620f0 admin admin] PATH_INFO: `/v3/users/806d6847d3fa4fcfb422601e8b285901` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:57.412214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e9c33-ce3b-4bb3-aade-2298095620f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=806d6847d3fa4fcfb422601e8b285901)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:57.415016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e9c33-ce3b-4bb3-aade-2298095620f0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:57.425818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fce54-737b-42c9-90cc-98057f23ac7a service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:57.427787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e15fce54-737b-42c9-90cc-98057f23ac7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:57.491471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fce54-737b-42c9-90cc-98057f23ac7a service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:57.513343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a40f2-0b4d-4faf-b92f-4effcc228169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:57.515532 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a40f2-0b4d-4faf-b92f-4effcc228169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:57.516033 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a40f2-0b4d-4faf-b92f-4effcc228169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:57.516510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a40f2-0b4d-4faf-b92f-4effcc228169 admin admin] PATH_INFO: `/v3/users/9a8e746ee67b45bd8d0988de9287407f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:57.525780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a40f2-0b4d-4faf-b92f-4effcc228169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8e746ee67b45bd8d0988de9287407f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:57.530412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a40f2-0b4d-4faf-b92f-4effcc228169 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:57.726411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:57.816108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6b7e9c33-ce3b-4bb3-aade-2298095620f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806d6847d3fa4fcfb422601e8b285901'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:57.819724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6b7e9c33-ce3b-4bb3-aade-2298095620f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806d6847d3fa4fcfb422601e8b285901'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:57.841982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6b7e9c33-ce3b-4bb3-aade-2298095620f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806d6847d3fa4fcfb422601e8b285901'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:57.917527 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ba0a40f2-0b4d-4faf-b92f-4effcc228169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8e746ee67b45bd8d0988de9287407f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:57.932274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ba0a40f2-0b4d-4faf-b92f-4effcc228169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8e746ee67b45bd8d0988de9287407f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:57.955183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6b7e9c33-ce3b-4bb3-aade-2298095620f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806d6847d3fa4fcfb422601e8b285901'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:57.956132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6b7e9c33-ce3b-4bb3-aade-2298095620f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806d6847d3fa4fcfb422601e8b285901'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:57.992567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 904/1790] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:57 2022] DELETE /identity/v3/users/806d6847d3fa4fcfb422601e8b285901 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:57.999574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e3c3b-1f2d-446d-8cf1-d172a5f34759 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:58.000392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e3c3b-1f2d-446d-8cf1-d172a5f34759 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:58.001070 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e3c3b-1f2d-446d-8cf1-d172a5f34759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:58.040958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ba0a40f2-0b4d-4faf-b92f-4effcc228169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8e746ee67b45bd8d0988de9287407f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:58.041862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ba0a40f2-0b4d-4faf-b92f-4effcc228169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8e746ee67b45bd8d0988de9287407f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:58.087095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ab3e3c3b-1f2d-446d-8cf1-d172a5f34759 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:58.092394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ab3e3c3b-1f2d-446d-8cf1-d172a5f34759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:58.111594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ba0a40f2-0b4d-4faf-b92f-4effcc228169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8e746ee67b45bd8d0988de9287407f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:12:58.138314 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 887/1791] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:57 2022] DELETE /identity/v3/users/9a8e746ee67b45bd8d0988de9287407f => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:12:58.148641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164d734-5ae0-46b2-a606-7aef47453b1a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:58.149219 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164d734-5ae0-46b2-a606-7aef47453b1a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:58.150817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164d734-5ae0-46b2-a606-7aef47453b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:58.216231 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d164d734-5ae0-46b2-a606-7aef47453b1a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:12:58.219692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d164d734-5ae0-46b2-a606-7aef47453b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:58.369209 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:58.413771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 905/1792] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:58.469932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e3c3b-1f2d-446d-8cf1-d172a5f34759 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:58.474861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ab3e3c3b-1f2d-446d-8cf1-d172a5f34759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:58.484344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 888/1793] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:12:58.498252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164d734-5ae0-46b2-a606-7aef47453b1a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:58.500781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d164d734-5ae0-46b2-a606-7aef47453b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:58.608003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e3c3b-1f2d-446d-8cf1-d172a5f34759 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:58.687609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164d734-5ae0-46b2-a606-7aef47453b1a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:58.692155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa993db-c755-4408-b526-83317d8447a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:58.693153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa993db-c755-4408-b526-83317d8447a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:58.693765 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa993db-c755-4408-b526-83317d8447a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:58.694217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa993db-c755-4408-b526-83317d8447a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:58.730445 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa993db-c755-4408-b526-83317d8447a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:58.732060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa993db-c755-4408-b526-83317d8447a0 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:58.776542 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 889/1794] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:12:58.828480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ae8e1-aca7-49b4-9b0b-b76884cfe161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:58.829276 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ae8e1-aca7-49b4-9b0b-b76884cfe161 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:58.829807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ae8e1-aca7-49b4-9b0b-b76884cfe161 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:58.830192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ae8e1-aca7-49b4-9b0b-b76884cfe161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:58.854926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ae8e1-aca7-49b4-9b0b-b76884cfe161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:58.856777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ae8e1-aca7-49b4-9b0b-b76884cfe161 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:12:58.888342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 906/1795] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:12:58.995363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:12:59.790914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa993db-c755-4408-b526-83317d8447a0 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:12:59.793104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7aa993db-c755-4408-b526-83317d8447a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:12:59.819419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa993db-c755-4408-b526-83317d8447a0 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:12:59.828845 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:12:59.829870 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:12:59.830367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:12:59.830843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] PATH_INFO: `/v3/users/4fa961aecf2d45ec98992f88c24857a2` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:12:59.833411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fa961aecf2d45ec98992f88c24857a2)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:12:59.835624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:00.123032 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fa961aecf2d45ec98992f88c24857a2'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:00.141524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fa961aecf2d45ec98992f88c24857a2'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:00.229333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fa961aecf2d45ec98992f88c24857a2'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:00.230191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fa961aecf2d45ec98992f88c24857a2'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:00.251795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fa961aecf2d45ec98992f88c24857a2'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:00.267735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 890/1796] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:12:59 2022] DELETE /identity/v3/users/4fa961aecf2d45ec98992f88c24857a2 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:00.275762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a1949-212f-41c3-9b7a-a57d64afc34e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:00.276295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a1949-212f-41c3-9b7a-a57d64afc34e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:00.276547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a1949-212f-41c3-9b7a-a57d64afc34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:00.355913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-798a1949-212f-41c3-9b7a-a57d64afc34e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:00.363516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-798a1949-212f-41c3-9b7a-a57d64afc34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:00.534735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 907/1797] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:00.553025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:00.555840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:00.609597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:00.702123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0faa9af-ad29-44ad-b2ea-e43bff084e89 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:00.726398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7434e5-6ffe-4d12-9f72-b4e2bbf5f46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:00.728586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7434e5-6ffe-4d12-9f72-b4e2bbf5f46c service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:00.728586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7434e5-6ffe-4d12-9f72-b4e2bbf5f46c service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:00.729210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7434e5-6ffe-4d12-9f72-b4e2bbf5f46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:00.753946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7434e5-6ffe-4d12-9f72-b4e2bbf5f46c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:00.756402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:00.758580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7434e5-6ffe-4d12-9f72-b4e2bbf5f46c service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:00.784706 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 891/1798] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:01.157849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a1949-212f-41c3-9b7a-a57d64afc34e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:01.159303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-798a1949-212f-41c3-9b7a-a57d64afc34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:01.165095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7434e5-6ffe-4d12-9f72-b4e2bbf5f46c service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:01.171460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4b7434e5-6ffe-4d12-9f72-b4e2bbf5f46c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:01.183711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a1949-212f-41c3-9b7a-a57d64afc34e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:01.194809 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:01.195602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:01.195887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:01.196163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] PATH_INFO: `/v3/projects/6b4da7db7b464b4f9f3e11b77b934db3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:01.196725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7434e5-6ffe-4d12-9f72-b4e2bbf5f46c service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:01.197572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b4da7db7b464b4f9f3e11b77b934db3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:01.203487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:01.203817 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:01.204791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:01.205596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:01.210062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] PATH_INFO: `/v3/projects/25a6cfb1dbc54355813127c34dc09380` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:01.210062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a6cfb1dbc54355813127c34dc09380)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:01.211104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:01.862087 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a6cfb1dbc54355813127c34dc09380', 'request_id': 'req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:01.862769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b4da7db7b464b4f9f3e11b77b934db3', 'request_id': 'req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:01.863382 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a6cfb1dbc54355813127c34dc09380', 'request_id': 'req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:01.863474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b4da7db7b464b4f9f3e11b77b934db3', 'request_id': 'req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:01.863474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b4da7db7b464b4f9f3e11b77b934db3', 'request_id': 'req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:02.566312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a6cfb1dbc54355813127c34dc09380', 'request_id': 'req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:02.579079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] The token cache is being invalidate because project 25a6cfb1dbc54355813127c34dc09380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:02.579461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:02.580876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:02.584813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 892/1799] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:01 2022] DELETE /identity/v3/projects/25a6cfb1dbc54355813127c34dc09380 => generated 0 bytes in 1421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:02.585924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:02.587717 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:02.681285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d53790-37c5-4c6a-9ff3-1f8d2455d41c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:02.707365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] The token cache is being invalidate because project 6b4da7db7b464b4f9f3e11b77b934db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:02.707791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:02.708899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-db7d6d00-9a33-4eeb-8e6b-63a158c7d180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:02.720054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 908/1800] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:01 2022] DELETE /identity/v3/projects/6b4da7db7b464b4f9f3e11b77b934db3 => generated 0 bytes in 1563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:02.723145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672720ac-e216-4977-9974-306ad43979c4 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:02.727087 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672720ac-e216-4977-9974-306ad43979c4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:02.729270 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672720ac-e216-4977-9974-306ad43979c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:02.787316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-672720ac-e216-4977-9974-306ad43979c4 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:02.793240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-672720ac-e216-4977-9974-306ad43979c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:02.861675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:02.865014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:02.865431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:02.866017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] PATH_INFO: `/v3/projects/bf3142a6a4524ff1986f13f92c1c6cb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:02.875486 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf3142a6a4524ff1986f13f92c1c6cb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:02.878004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:02.915513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 909/1801] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:02.916623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672720ac-e216-4977-9974-306ad43979c4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:02.918324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-672720ac-e216-4977-9974-306ad43979c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:02.945317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672720ac-e216-4977-9974-306ad43979c4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:02.956996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:02.958512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:02.959008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:02.959502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] PATH_INFO: `/v3/users/e9cf8119e3084450bd33f117fc2d9089` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:02.961970 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9cf8119e3084450bd33f117fc2d9089)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:02.965717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:03.099821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf3142a6a4524ff1986f13f92c1c6cb5', 'request_id': 'req-6c9a4f7c-9907-4b15-a352-b55ff8030d51'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:03.112020 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf3142a6a4524ff1986f13f92c1c6cb5', 'request_id': 'req-6c9a4f7c-9907-4b15-a352-b55ff8030d51'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:03.227641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf3142a6a4524ff1986f13f92c1c6cb5', 'request_id': 'req-6c9a4f7c-9907-4b15-a352-b55ff8030d51'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:03.257185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] The token cache is being invalidate because project bf3142a6a4524ff1986f13f92c1c6cb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:03.258027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:03.258737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-6c9a4f7c-9907-4b15-a352-b55ff8030d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:03.269703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 893/1802] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:02 2022] DELETE /identity/v3/projects/bf3142a6a4524ff1986f13f92c1c6cb5 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:03.269703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a1011-68d9-4687-8ece-4c6075d750b1 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:03.269703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a1011-68d9-4687-8ece-4c6075d750b1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:03.269703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a1011-68d9-4687-8ece-4c6075d750b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:03.270149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9cf8119e3084450bd33f117fc2d9089'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:03.274248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9cf8119e3084450bd33f117fc2d9089'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:03.292129 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-261a1011-68d9-4687-8ece-4c6075d750b1 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:03.293753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-261a1011-68d9-4687-8ece-4c6075d750b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:03.311941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9cf8119e3084450bd33f117fc2d9089'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:03.447444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9cf8119e3084450bd33f117fc2d9089'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:03.448092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9cf8119e3084450bd33f117fc2d9089'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:03.477038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 894/1803] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:03.477829 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a1011-68d9-4687-8ece-4c6075d750b1 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:03.483990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-261a1011-68d9-4687-8ece-4c6075d750b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:03.484182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 910/1804] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:02 2022] DELETE /identity/v3/users/e9cf8119e3084450bd33f117fc2d9089 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:03.485162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:03.505920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a1011-68d9-4687-8ece-4c6075d750b1 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:03.598845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443825d6-b13c-4e36-8850-eaabb9d692b1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:03.605261 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24213b8c-f4ee-421e-b5f6-37a82ed7450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:03.605958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24213b8c-f4ee-421e-b5f6-37a82ed7450e admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:03.606262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24213b8c-f4ee-421e-b5f6-37a82ed7450e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:03.606671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24213b8c-f4ee-421e-b5f6-37a82ed7450e admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:03.608124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24213b8c-f4ee-421e-b5f6-37a82ed7450e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:13:03.608428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24213b8c-f4ee-421e-b5f6-37a82ed7450e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:03.610657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9c79e-c8bb-4927-a646-f1cd84807465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:03.611165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24213b8c-f4ee-421e-b5f6-37a82ed7450e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:03.611447 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c79e-c8bb-4927-a646-f1cd84807465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:03.611946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c79e-c8bb-4927-a646-f1cd84807465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:03.612172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c79e-c8bb-4927-a646-f1cd84807465 admin admin] PATH_INFO: `/v3/users/6432b726eb34478cbf8d57e2acaea12f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:03.613458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9c79e-c8bb-4927-a646-f1cd84807465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6432b726eb34478cbf8d57e2acaea12f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:03.615627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9c79e-c8bb-4927-a646-f1cd84807465 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:03.630804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 911/1805] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:13:03.634671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24213b8c-f4ee-421e-b5f6-37a82ed7450e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:03.648746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24213b8c-f4ee-421e-b5f6-37a82ed7450e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:03.654310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159e822-f046-4278-a07f-773db6c4e1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:03.654936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159e822-f046-4278-a07f-773db6c4e1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:03.655221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159e822-f046-4278-a07f-773db6c4e1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:03.655496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159e822-f046-4278-a07f-773db6c4e1e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:03.656168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159e822-f046-4278-a07f-773db6c4e1e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:03.658154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159e822-f046-4278-a07f-773db6c4e1e5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:03.793896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 912/1806] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:03 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:03.798545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159e822-f046-4278-a07f-773db6c4e1e5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:03.914124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159e822-f046-4278-a07f-773db6c4e1e5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:03.921722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34158c35-8f19-4372-953b-416a846bcce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:03.922793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34158c35-8f19-4372-953b-416a846bcce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:03.923233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34158c35-8f19-4372-953b-416a846bcce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:03.923511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34158c35-8f19-4372-953b-416a846bcce0 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:03.924193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34158c35-8f19-4372-953b-416a846bcce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:03.927051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34158c35-8f19-4372-953b-416a846bcce0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:03.933830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-00a9c79e-c8bb-4927-a646-f1cd84807465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6432b726eb34478cbf8d57e2acaea12f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:03.953123 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-00a9c79e-c8bb-4927-a646-f1cd84807465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6432b726eb34478cbf8d57e2acaea12f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:04.051871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-00a9c79e-c8bb-4927-a646-f1cd84807465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6432b726eb34478cbf8d57e2acaea12f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:04.052445 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-00a9c79e-c8bb-4927-a646-f1cd84807465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6432b726eb34478cbf8d57e2acaea12f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:04.064231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 913/1807] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:03 2022] POST /identity/v3/users => generated 289 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:04.065856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933c1bb-14fa-44d4-bfea-526ff858080c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:04.066900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933c1bb-14fa-44d4-bfea-526ff858080c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:04.067318 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933c1bb-14fa-44d4-bfea-526ff858080c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:04.069999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 914/1808] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:13:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:13:04.071272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933c1bb-14fa-44d4-bfea-526ff858080c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:04.087410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-00a9c79e-c8bb-4927-a646-f1cd84807465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6432b726eb34478cbf8d57e2acaea12f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:04.095392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4933c1bb-14fa-44d4-bfea-526ff858080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:04.106069 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 895/1809] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:03 2022] DELETE /identity/v3/users/6432b726eb34478cbf8d57e2acaea12f => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:04.106069 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557fe75-2375-405e-af76-a1389f97d6f7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:04.106069 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557fe75-2375-405e-af76-a1389f97d6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:04.106500 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557fe75-2375-405e-af76-a1389f97d6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:04.121483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933c1bb-14fa-44d4-bfea-526ff858080c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:04.139225 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5557fe75-2375-405e-af76-a1389f97d6f7 None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:04.140223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5557fe75-2375-405e-af76-a1389f97d6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:04.209495 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30701097-2e01-41a7-b909-9f26d63d56a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:04.211236 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30701097-2e01-41a7-b909-9f26d63d56a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:04.211711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30701097-2e01-41a7-b909-9f26d63d56a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:04.212128 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30701097-2e01-41a7-b909-9f26d63d56a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:04.213206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30701097-2e01-41a7-b909-9f26d63d56a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:04.215699 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30701097-2e01-41a7-b909-9f26d63d56a5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:04.238390 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 915/1810] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:04.242150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30701097-2e01-41a7-b909-9f26d63d56a5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:04.254024 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 896/1811] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:13:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:04.260063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30701097-2e01-41a7-b909-9f26d63d56a5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:04.270914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27bf58-6952-4d64-972c-36b94fa0c10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:04.271675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27bf58-6952-4d64-972c-36b94fa0c10c admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:04.272067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27bf58-6952-4d64-972c-36b94fa0c10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:04.272724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27bf58-6952-4d64-972c-36b94fa0c10c admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:04.273944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27bf58-6952-4d64-972c-36b94fa0c10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:04.276294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27bf58-6952-4d64-972c-36b94fa0c10c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:04.290697 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 916/1812] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:13:04.296566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557fe75-2375-405e-af76-a1389f97d6f7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:04.310954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557fe75-2375-405e-af76-a1389f97d6f7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:04.316126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:04.318025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22031f-d15d-4874-a6b2-e3753edb821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:04.319136 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22031f-d15d-4874-a6b2-e3753edb821b admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:04.319531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22031f-d15d-4874-a6b2-e3753edb821b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:04.320237 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22031f-d15d-4874-a6b2-e3753edb821b admin admin] PATH_INFO: `/v3/projects/6de15323f5af42d493b9f6f9dcd69b52/users/6e9e73d29ed64bad8394004fec02e16d/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:04.356307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22031f-d15d-4874-a6b2-e3753edb821b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de15323f5af42d493b9f6f9dcd69b52, user_id=6e9e73d29ed64bad8394004fec02e16d, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:04.359639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22031f-d15d-4874-a6b2-e3753edb821b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:04.444076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 897/1813] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:04 2022] PUT /identity/v3/projects/6de15323f5af42d493b9f6f9dcd69b52/users/6e9e73d29ed64bad8394004fec02e16d/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:04.457499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7716-761d-4395-9b59-6b31dbe73189 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:04.457787 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7716-761d-4395-9b59-6b31dbe73189 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:04.458411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7716-761d-4395-9b59-6b31dbe73189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:04.508063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e6ba7716-761d-4395-9b59-6b31dbe73189 None None] MFA Rules not processed for user `6e9e73d29ed64bad8394004fec02e16d`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:04.509647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e6ba7716-761d-4395-9b59-6b31dbe73189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:04.569417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:04.587251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 917/1814] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:04.599131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22031f-d15d-4874-a6b2-e3753edb821b admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:04.601339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3e22031f-d15d-4874-a6b2-e3753edb821b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:04.614436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba7716-761d-4395-9b59-6b31dbe73189 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:04.615840 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e6ba7716-761d-4395-9b59-6b31dbe73189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:04.704875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22031f-d15d-4874-a6b2-e3753edb821b admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:04.708526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eebc73-5051-49ff-b16b-759b62f69092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:04.709323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eebc73-5051-49ff-b16b-759b62f69092 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:04.709619 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eebc73-5051-49ff-b16b-759b62f69092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:04.710643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eebc73-5051-49ff-b16b-759b62f69092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:04.723565 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eebc73-5051-49ff-b16b-759b62f69092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:04.725744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eebc73-5051-49ff-b16b-759b62f69092 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:04.739356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 898/1815] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:04.752484 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba7716-761d-4395-9b59-6b31dbe73189 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:04.760696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eebc73-5051-49ff-b16b-759b62f69092 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:04.770988 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a8eebc73-5051-49ff-b16b-759b62f69092 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:04.771698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:04.771698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:04.771698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:04.771698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] PATH_INFO: `/v3/projects/1d3f486da27d4806a0e016f8f265a486` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:04.772049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d3f486da27d4806a0e016f8f265a486)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:04.772049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:04.970216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eebc73-5051-49ff-b16b-759b62f69092 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:05.005696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d3f486da27d4806a0e016f8f265a486', 'request_id': 'req-c9e97338-e500-41f6-9364-8d81644ca44e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:05.025206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d3f486da27d4806a0e016f8f265a486', 'request_id': 'req-c9e97338-e500-41f6-9364-8d81644ca44e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:05.038060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d3f486da27d4806a0e016f8f265a486', 'request_id': 'req-c9e97338-e500-41f6-9364-8d81644ca44e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:05.102051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a60fd-68ab-480e-a9d3-f457cb5d3913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:05.102942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a60fd-68ab-480e-a9d3-f457cb5d3913 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:05.103385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a60fd-68ab-480e-a9d3-f457cb5d3913 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:05.103816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a60fd-68ab-480e-a9d3-f457cb5d3913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:05.118635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a60fd-68ab-480e-a9d3-f457cb5d3913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:05.121312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a60fd-68ab-480e-a9d3-f457cb5d3913 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:05.134980 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 899/1816] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:13:05.193799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] The token cache is being invalidate because project 1d3f486da27d4806a0e016f8f265a486 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:05.194193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:05.197744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:05.205914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 918/1817] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:04 2022] DELETE /identity/v3/projects/1d3f486da27d4806a0e016f8f265a486 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:05.207876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e8b80-318a-4491-9529-53b950f313f0 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:05.208341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e8b80-318a-4491-9529-53b950f313f0 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:05.208785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e8b80-318a-4491-9529-53b950f313f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:05.281581 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7a9e8b80-318a-4491-9529-53b950f313f0 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:05.284025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7a9e8b80-318a-4491-9529-53b950f313f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:05.318187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:05.319764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:05.402761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 900/1818] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:05.411398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b5e83-2db8-428d-9519-87aa08f7f8ba None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:05.411816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b5e83-2db8-428d-9519-87aa08f7f8ba None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:05.412349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b5e83-2db8-428d-9519-87aa08f7f8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:05.491433 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e97338-e500-41f6-9364-8d81644ca44e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:05.498396 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:05.499423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:05.499927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:05.500446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] PATH_INFO: `/v3/projects/d25fdc3d0497443aa8a4762d03f8abcd` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:05.501920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d25fdc3d0497443aa8a4762d03f8abcd)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:05.504135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:05.513801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-729b5e83-2db8-428d-9519-87aa08f7f8ba None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:05.513801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-729b5e83-2db8-428d-9519-87aa08f7f8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:05.529997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 901/1819] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:05.534391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b5e83-2db8-428d-9519-87aa08f7f8ba None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:05.568953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b5e83-2db8-428d-9519-87aa08f7f8ba None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:05.586105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e508f7-bb16-405d-9944-25a2d7a8dd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:05.586589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e508f7-bb16-405d-9944-25a2d7a8dd8a admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:05.587438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e508f7-bb16-405d-9944-25a2d7a8dd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:05.587731 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e508f7-bb16-405d-9944-25a2d7a8dd8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:05.588477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e508f7-bb16-405d-9944-25a2d7a8dd8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:13:05.588774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e508f7-bb16-405d-9944-25a2d7a8dd8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:05.590737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e508f7-bb16-405d-9944-25a2d7a8dd8a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:05.613918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 902/1820] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:13:05.613918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e508f7-bb16-405d-9944-25a2d7a8dd8a admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:05.625226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e508f7-bb16-405d-9944-25a2d7a8dd8a admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:05.630508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f3524-9be0-499b-8bc0-0e7307e1e0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:05.631297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f3524-9be0-499b-8bc0-0e7307e1e0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:05.631587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f3524-9be0-499b-8bc0-0e7307e1e0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:05.631864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f3524-9be0-499b-8bc0-0e7307e1e0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:05.632556 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f3524-9be0-499b-8bc0-0e7307e1e0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:05.635327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f3524-9be0-499b-8bc0-0e7307e1e0a1 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:05.674944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd25fdc3d0497443aa8a4762d03f8abcd', 'request_id': 'req-338a4a77-3c1c-4c4e-a29f-df44011f4875'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:05.697542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd25fdc3d0497443aa8a4762d03f8abcd', 'request_id': 'req-338a4a77-3c1c-4c4e-a29f-df44011f4875'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:05.709878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd25fdc3d0497443aa8a4762d03f8abcd', 'request_id': 'req-338a4a77-3c1c-4c4e-a29f-df44011f4875'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:05.746914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 903/1821] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:05 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:13:05.762024 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f3524-9be0-499b-8bc0-0e7307e1e0a1 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:05.818597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] The token cache is being invalidate because project d25fdc3d0497443aa8a4762d03f8abcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:05.819141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:05.819963 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-338a4a77-3c1c-4c4e-a29f-df44011f4875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:05.824981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 919/1822] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:05 2022] DELETE /identity/v3/projects/d25fdc3d0497443aa8a4762d03f8abcd => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:05.827186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71573cce-2a2a-41b8-9cd3-a675c0a0bea7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:05.827719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71573cce-2a2a-41b8-9cd3-a675c0a0bea7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:05.827835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71573cce-2a2a-41b8-9cd3-a675c0a0bea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:05.844866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f3524-9be0-499b-8bc0-0e7307e1e0a1 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:05.847811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b233afa-3f05-4bad-b3cb-820c4a8317fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:05.848580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233afa-3f05-4bad-b3cb-820c4a8317fe admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:05.849031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233afa-3f05-4bad-b3cb-820c4a8317fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:05.849382 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233afa-3f05-4bad-b3cb-820c4a8317fe admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:05.850266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b233afa-3f05-4bad-b3cb-820c4a8317fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:05.852338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b233afa-3f05-4bad-b3cb-820c4a8317fe admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:05.856359 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-71573cce-2a2a-41b8-9cd3-a675c0a0bea7 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:05.857892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-71573cce-2a2a-41b8-9cd3-a675c0a0bea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:05.877304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 920/1823] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:05.879120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cf4bc-fe6c-4595-af79-77ef2941b3f5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:05.879558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cf4bc-fe6c-4595-af79-77ef2941b3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:05.880137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cf4bc-fe6c-4595-af79-77ef2941b3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:05.905831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e44cf4bc-fe6c-4595-af79-77ef2941b3f5 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:05.907158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e44cf4bc-fe6c-4595-af79-77ef2941b3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:05.915250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 921/1824] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:05.919518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cf4bc-fe6c-4595-af79-77ef2941b3f5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:05.929438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cf4bc-fe6c-4595-af79-77ef2941b3f5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:05.938442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464878d8-5b30-40d2-a379-ce3221c95328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:05.939306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464878d8-5b30-40d2-a379-ce3221c95328 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:05.939594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464878d8-5b30-40d2-a379-ce3221c95328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:05.939863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464878d8-5b30-40d2-a379-ce3221c95328 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:05.940739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464878d8-5b30-40d2-a379-ce3221c95328 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:13:05.941092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464878d8-5b30-40d2-a379-ce3221c95328 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:05.943517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464878d8-5b30-40d2-a379-ce3221c95328 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:05.977677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 922/1825] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:13:05.981899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464878d8-5b30-40d2-a379-ce3221c95328 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.007135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 904/1826] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:05 2022] POST /identity/v3/users => generated 283 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:06.010363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b233afa-3f05-4bad-b3cb-820c4a8317fe admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.018244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5b233afa-3f05-4bad-b3cb-820c4a8317fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:06.026873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464878d8-5b30-40d2-a379-ce3221c95328 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.042012 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba78530-2505-4a51-8ebd-cdbb2daf3141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.043080 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba78530-2505-4a51-8ebd-cdbb2daf3141 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.043501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba78530-2505-4a51-8ebd-cdbb2daf3141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.047651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba78530-2505-4a51-8ebd-cdbb2daf3141 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.048598 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba78530-2505-4a51-8ebd-cdbb2daf3141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.050673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba78530-2505-4a51-8ebd-cdbb2daf3141 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.108290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b233afa-3f05-4bad-b3cb-820c4a8317fe admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.119655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89d7a8-eb59-4916-8e1b-1873234820e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.120545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89d7a8-eb59-4916-8e1b-1873234820e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.121149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89d7a8-eb59-4916-8e1b-1873234820e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.121580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89d7a8-eb59-4916-8e1b-1873234820e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.123095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89d7a8-eb59-4916-8e1b-1873234820e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.125013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89d7a8-eb59-4916-8e1b-1873234820e0 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.139814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 905/1827] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:06.145010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89d7a8-eb59-4916-8e1b-1873234820e0 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.161335 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89d7a8-eb59-4916-8e1b-1873234820e0 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.167011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e527dd0-24d1-404e-a241-72e31c15c66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.179733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e527dd0-24d1-404e-a241-72e31c15c66a admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.179733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e527dd0-24d1-404e-a241-72e31c15c66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.179733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e527dd0-24d1-404e-a241-72e31c15c66a admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.179733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e527dd0-24d1-404e-a241-72e31c15c66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.180244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 923/1828] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:05 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:06.181785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e527dd0-24d1-404e-a241-72e31c15c66a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.189061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba78530-2505-4a51-8ebd-cdbb2daf3141 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.207285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 906/1829] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:13:06.211254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e527dd0-24d1-404e-a241-72e31c15c66a admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.231120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e527dd0-24d1-404e-a241-72e31c15c66a admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.306871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba78530-2505-4a51-8ebd-cdbb2daf3141 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.319198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff869b5-2cc5-483f-824a-b4dc63af1b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.323825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff869b5-2cc5-483f-824a-b4dc63af1b10 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.324014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff869b5-2cc5-483f-824a-b4dc63af1b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.324364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff869b5-2cc5-483f-824a-b4dc63af1b10 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.325195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff869b5-2cc5-483f-824a-b4dc63af1b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.327978 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff869b5-2cc5-483f-824a-b4dc63af1b10 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.361821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc1fa9-3892-4152-97cf-38b9d87ba63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.362811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1fa9-3892-4152-97cf-38b9d87ba63e admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.363166 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1fa9-3892-4152-97cf-38b9d87ba63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.363515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1fa9-3892-4152-97cf-38b9d87ba63e admin admin] PATH_INFO: `/v3/projects/57de1e0db9be4cc2ac06c5636fe44c2e/users/f93cfa81261744259dc1075307bd9ca4/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.386132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc1fa9-3892-4152-97cf-38b9d87ba63e admin admin] RBAC: Authorizing `identity:create_grant(project_id=57de1e0db9be4cc2ac06c5636fe44c2e, user_id=f93cfa81261744259dc1075307bd9ca4, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.389738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc1fa9-3892-4152-97cf-38b9d87ba63e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.423220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 924/1830] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:06 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:06.424202 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff869b5-2cc5-483f-824a-b4dc63af1b10 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.434573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff869b5-2cc5-483f-824a-b4dc63af1b10 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.439901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35000d-d02a-4300-8c1d-f3073444b951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.440532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35000d-d02a-4300-8c1d-f3073444b951 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.440820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35000d-d02a-4300-8c1d-f3073444b951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.441133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35000d-d02a-4300-8c1d-f3073444b951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.441836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35000d-d02a-4300-8c1d-f3073444b951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.443451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35000d-d02a-4300-8c1d-f3073444b951 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.457017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 907/1831] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:06 2022] PUT /identity/v3/projects/57de1e0db9be4cc2ac06c5636fe44c2e/users/f93cfa81261744259dc1075307bd9ca4/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:06.457723 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 925/1832] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:06.459445 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc1fa9-3892-4152-97cf-38b9d87ba63e admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.470217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090df1a0-4b3f-46ef-abd8-ddb40dbb900d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.470508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090df1a0-4b3f-46ef-abd8-ddb40dbb900d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.477559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090df1a0-4b3f-46ef-abd8-ddb40dbb900d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.506864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-090df1a0-4b3f-46ef-abd8-ddb40dbb900d None None] MFA Rules not processed for user `f93cfa81261744259dc1075307bd9ca4`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:06.506864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-090df1a0-4b3f-46ef-abd8-ddb40dbb900d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:06.527172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc1fa9-3892-4152-97cf-38b9d87ba63e admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.532606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafcac3-6f9e-4df3-bd76-d6f24bd03673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.533284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafcac3-6f9e-4df3-bd76-d6f24bd03673 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.535076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafcac3-6f9e-4df3-bd76-d6f24bd03673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.535076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafcac3-6f9e-4df3-bd76-d6f24bd03673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.535076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafcac3-6f9e-4df3-bd76-d6f24bd03673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.536358 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafcac3-6f9e-4df3-bd76-d6f24bd03673 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.547025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 908/1833] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:13:06.551140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafcac3-6f9e-4df3-bd76-d6f24bd03673 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.561902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafcac3-6f9e-4df3-bd76-d6f24bd03673 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.566932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150b127-9de0-4195-92e2-50c4b95df79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.567684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150b127-9de0-4195-92e2-50c4b95df79a admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.568042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150b127-9de0-4195-92e2-50c4b95df79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.568396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150b127-9de0-4195-92e2-50c4b95df79a admin admin] PATH_INFO: `/v3/projects/c1090bdab94a46a1b5dcf823d9b1c4fc/users/91f74fecab664aeba143108777ee02f0/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.585018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 926/1834] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:06.588494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090df1a0-4b3f-46ef-abd8-ddb40dbb900d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.599439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090df1a0-4b3f-46ef-abd8-ddb40dbb900d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.600026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150b127-9de0-4195-92e2-50c4b95df79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1090bdab94a46a1b5dcf823d9b1c4fc, user_id=91f74fecab664aeba143108777ee02f0, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.604315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38ca55-158a-4704-a0f2-6d6a17fe2f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.604944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38ca55-158a-4704-a0f2-6d6a17fe2f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.605268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38ca55-158a-4704-a0f2-6d6a17fe2f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.605539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38ca55-158a-4704-a0f2-6d6a17fe2f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.606239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38ca55-158a-4704-a0f2-6d6a17fe2f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.608221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38ca55-158a-4704-a0f2-6d6a17fe2f40 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.610555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150b127-9de0-4195-92e2-50c4b95df79a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.709219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 927/1835] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:06 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:06.709441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 909/1836] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:06 2022] PUT /identity/v3/projects/c1090bdab94a46a1b5dcf823d9b1c4fc/users/91f74fecab664aeba143108777ee02f0/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:06.714240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38ca55-158a-4704-a0f2-6d6a17fe2f40 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.719174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645d04f-9d5d-40c7-8228-78b17576720f None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.719463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645d04f-9d5d-40c7-8228-78b17576720f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.720320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645d04f-9d5d-40c7-8228-78b17576720f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.768706 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f645d04f-9d5d-40c7-8228-78b17576720f None None] MFA Rules not processed for user `91f74fecab664aeba143108777ee02f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:06.770073 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f645d04f-9d5d-40c7-8228-78b17576720f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:06.804242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38ca55-158a-4704-a0f2-6d6a17fe2f40 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.810910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d712aad-737b-4136-ba73-23c7f4e56f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.811920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d712aad-737b-4136-ba73-23c7f4e56f24 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.812306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d712aad-737b-4136-ba73-23c7f4e56f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.812711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d712aad-737b-4136-ba73-23c7f4e56f24 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.813520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d712aad-737b-4136-ba73-23c7f4e56f24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.816241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d712aad-737b-4136-ba73-23c7f4e56f24 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.871996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 910/1837] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:13:06.876600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645d04f-9d5d-40c7-8228-78b17576720f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.889391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645d04f-9d5d-40c7-8228-78b17576720f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.894637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a8126-120d-4cf2-b367-03e9f2802486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.895795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a8126-120d-4cf2-b367-03e9f2802486 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.895795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a8126-120d-4cf2-b367-03e9f2802486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.896118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a8126-120d-4cf2-b367-03e9f2802486 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.896843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a8126-120d-4cf2-b367-03e9f2802486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.898989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a8126-120d-4cf2-b367-03e9f2802486 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.954111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 928/1838] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:06 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:06.955224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d712aad-737b-4136-ba73-23c7f4e56f24 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.964787 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d712aad-737b-4136-ba73-23c7f4e56f24 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:06.970463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad5e95-f310-4f74-aee1-920afeeeba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:06.971078 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5e95-f310-4f74-aee1-920afeeeba93 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:06.971410 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5e95-f310-4f74-aee1-920afeeeba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:06.971635 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5e95-f310-4f74-aee1-920afeeeba93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:06.972334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad5e95-f310-4f74-aee1-920afeeeba93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:06.973945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad5e95-f310-4f74-aee1-920afeeeba93 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:06.985166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 929/1839] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:06.989281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad5e95-f310-4f74-aee1-920afeeeba93 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:06.999276 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad5e95-f310-4f74-aee1-920afeeeba93 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:07.003974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae61f3-935e-4ab8-9ac4-6020349cdea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:07.004632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae61f3-935e-4ab8-9ac4-6020349cdea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:07.004915 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae61f3-935e-4ab8-9ac4-6020349cdea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:07.005226 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae61f3-935e-4ab8-9ac4-6020349cdea7 admin admin] PATH_INFO: `/v3/projects/3ff6e6037d874d9d817bf13b432c98d8/users/ceea33185b884d1aa503b479b6d530e8/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:07.007677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:07.021450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae61f3-935e-4ab8-9ac4-6020349cdea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff6e6037d874d9d817bf13b432c98d8, user_id=ceea33185b884d1aa503b479b6d530e8, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:07.024083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae61f3-935e-4ab8-9ac4-6020349cdea7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:07.071740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 911/1840] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:06 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:07.076994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a8126-120d-4cf2-b367-03e9f2802486 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:07.106876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 930/1841] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:06 2022] PUT /identity/v3/projects/3ff6e6037d874d9d817bf13b432c98d8/users/ceea33185b884d1aa503b479b6d530e8/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:07.121575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae61f3-935e-4ab8-9ac4-6020349cdea7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:07.163217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a8126-120d-4cf2-b367-03e9f2802486 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:07.281172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c59fb-b21b-45f5-836f-ee3780a8c71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:07.281826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c59fb-b21b-45f5-836f-ee3780a8c71d admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:07.282116 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c59fb-b21b-45f5-836f-ee3780a8c71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:07.282396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c59fb-b21b-45f5-836f-ee3780a8c71d admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:07.283297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c59fb-b21b-45f5-836f-ee3780a8c71d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:07.285252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c59fb-b21b-45f5-836f-ee3780a8c71d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:07.285399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae61f3-935e-4ab8-9ac4-6020349cdea7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:07.297699 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72a270-73ac-4e70-b711-a47448211ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:07.303272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72a270-73ac-4e70-b711-a47448211ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:07.303272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72a270-73ac-4e70-b711-a47448211ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:07.303272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72a270-73ac-4e70-b711-a47448211ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:07.303272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72a270-73ac-4e70-b711-a47448211ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:07.303272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72a270-73ac-4e70-b711-a47448211ecd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:07.315655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 931/1842] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:07.319604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72a270-73ac-4e70-b711-a47448211ecd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:07.343562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72a270-73ac-4e70-b711-a47448211ecd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:07.351654 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bbcfd-557f-44ce-b8a5-8732dbf56d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:07.352477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bbcfd-557f-44ce-b8a5-8732dbf56d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:07.352858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bbcfd-557f-44ce-b8a5-8732dbf56d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:07.353454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bbcfd-557f-44ce-b8a5-8732dbf56d47 admin admin] PATH_INFO: `/v3/projects/3ff6e6037d874d9d817bf13b432c98d8/users/ceea33185b884d1aa503b479b6d530e8/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:07.356447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bbcfd-557f-44ce-b8a5-8732dbf56d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff6e6037d874d9d817bf13b432c98d8, user_id=ceea33185b884d1aa503b479b6d530e8, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:07.361672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bbcfd-557f-44ce-b8a5-8732dbf56d47 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:07.389019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 932/1843] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:07 2022] PUT /identity/v3/projects/3ff6e6037d874d9d817bf13b432c98d8/users/ceea33185b884d1aa503b479b6d530e8/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:07.392359 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e9b0d-4e73-44e3-8785-e57e2182f44a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:07.394320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e9b0d-4e73-44e3-8785-e57e2182f44a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:07.394858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e9b0d-4e73-44e3-8785-e57e2182f44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:07.419448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 912/1844] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:07 2022] POST /identity/v3/users => generated 300 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:13:07.421867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c59fb-b21b-45f5-836f-ee3780a8c71d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:07.449918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c59fb-b21b-45f5-836f-ee3780a8c71d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:07.451963 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f59e9b0d-4e73-44e3-8785-e57e2182f44a None None] MFA Rules not processed for user `ceea33185b884d1aa503b479b6d530e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:07.453258 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f59e9b0d-4e73-44e3-8785-e57e2182f44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:07.548463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e211e3-b654-4eee-8f08-a104e82cb077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:07.549296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e211e3-b654-4eee-8f08-a104e82cb077 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:07.549676 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e211e3-b654-4eee-8f08-a104e82cb077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:07.550152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e211e3-b654-4eee-8f08-a104e82cb077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:07.551842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e211e3-b654-4eee-8f08-a104e82cb077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:07.568602 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e211e3-b654-4eee-8f08-a104e82cb077 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:07.592356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 933/1845] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:07.597754 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 913/1846] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:07.597897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e9b0d-4e73-44e3-8785-e57e2182f44a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:07.609411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e9b0d-4e73-44e3-8785-e57e2182f44a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:07.614383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13fcb31-c42e-4b9f-a34b-e469f06dad7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:07.615085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fcb31-c42e-4b9f-a34b-e469f06dad7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:07.615368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fcb31-c42e-4b9f-a34b-e469f06dad7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:07.616004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fcb31-c42e-4b9f-a34b-e469f06dad7e admin admin] PATH_INFO: `/v3/projects/d6343f8e45a849f08eb6e98e2f35203b/users/606f4471e9834e57ad1d21cbc8859236/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:07.639900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13fcb31-c42e-4b9f-a34b-e469f06dad7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6343f8e45a849f08eb6e98e2f35203b, user_id=606f4471e9834e57ad1d21cbc8859236, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:07.642641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13fcb31-c42e-4b9f-a34b-e469f06dad7e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:07.759527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 934/1847] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:07 2022] PUT /identity/v3/projects/d6343f8e45a849f08eb6e98e2f35203b/users/606f4471e9834e57ad1d21cbc8859236/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:07.764175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e211e3-b654-4eee-8f08-a104e82cb077 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:07.776263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e211e3-b654-4eee-8f08-a104e82cb077 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:07.862351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70b915-4b04-467e-b283-a22d1989d82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:07.863398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70b915-4b04-467e-b283-a22d1989d82b admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:07.863398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70b915-4b04-467e-b283-a22d1989d82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:07.863731 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70b915-4b04-467e-b283-a22d1989d82b admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:07.864511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70b915-4b04-467e-b283-a22d1989d82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:07.869992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70b915-4b04-467e-b283-a22d1989d82b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:07.891806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 914/1848] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:07.893606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13fcb31-c42e-4b9f-a34b-e469f06dad7e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:07.899647 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70b915-4b04-467e-b283-a22d1989d82b admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:07.901598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-de70b915-4b04-467e-b283-a22d1989d82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:07.908699 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13fcb31-c42e-4b9f-a34b-e469f06dad7e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:07.916107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340d0f4-2ba7-4911-b010-65939eb18982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:07.916815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340d0f4-2ba7-4911-b010-65939eb18982 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:07.917426 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340d0f4-2ba7-4911-b010-65939eb18982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:07.917708 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340d0f4-2ba7-4911-b010-65939eb18982 admin admin] PATH_INFO: `/v3/projects/d6343f8e45a849f08eb6e98e2f35203b/users/606f4471e9834e57ad1d21cbc8859236/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:07.921719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340d0f4-2ba7-4911-b010-65939eb18982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6343f8e45a849f08eb6e98e2f35203b, user_id=606f4471e9834e57ad1d21cbc8859236, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:07.924355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340d0f4-2ba7-4911-b010-65939eb18982 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:07.941016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 935/1849] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:07 2022] PUT /identity/v3/projects/d6343f8e45a849f08eb6e98e2f35203b/users/606f4471e9834e57ad1d21cbc8859236/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:07.958900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa13ca4-3f9a-4a6b-8f4b-d7475c7dfb37 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:07.959357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa13ca4-3f9a-4a6b-8f4b-d7475c7dfb37 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:07.960373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa13ca4-3f9a-4a6b-8f4b-d7475c7dfb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:07.974271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70b915-4b04-467e-b283-a22d1989d82b admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:08.006661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:08.011644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-daa13ca4-3f9a-4a6b-8f4b-d7475c7dfb37 None None] MFA Rules not processed for user `606f4471e9834e57ad1d21cbc8859236`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:08.012899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-daa13ca4-3f9a-4a6b-8f4b-d7475c7dfb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:08.073873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15b543-bebf-4674-9973-a9bf8da8285b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:08.074514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b543-bebf-4674-9973-a9bf8da8285b service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:08.074803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b543-bebf-4674-9973-a9bf8da8285b service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:08.075081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b543-bebf-4674-9973-a9bf8da8285b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:08.177315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 936/1850] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:08.215598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15b543-bebf-4674-9973-a9bf8da8285b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:08.218545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15b543-bebf-4674-9973-a9bf8da8285b service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:08.250927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 915/1851] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:13:08.307328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa13ca4-3f9a-4a6b-8f4b-d7475c7dfb37 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:08.309060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-daa13ca4-3f9a-4a6b-8f4b-d7475c7dfb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:08.326698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa13ca4-3f9a-4a6b-8f4b-d7475c7dfb37 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:08.333441 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66432396-54f7-4701-baf4-e7a085f56296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:08.335106 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66432396-54f7-4701-baf4-e7a085f56296 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:08.335450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66432396-54f7-4701-baf4-e7a085f56296 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:08.335711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66432396-54f7-4701-baf4-e7a085f56296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:08.348818 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66432396-54f7-4701-baf4-e7a085f56296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:08.350782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66432396-54f7-4701-baf4-e7a085f56296 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:08.365054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 937/1852] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:08.488333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15b543-bebf-4674-9973-a9bf8da8285b service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:08.529193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66432396-54f7-4701-baf4-e7a085f56296 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:08.537367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66432396-54f7-4701-baf4-e7a085f56296 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:08.541120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeeb1f-0678-412b-a868-87454d3b2f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:08.541730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeeb1f-0678-412b-a868-87454d3b2f74 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:08.542037 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeeb1f-0678-412b-a868-87454d3b2f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:08.542276 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeeb1f-0678-412b-a868-87454d3b2f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:08.566181 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15b543-bebf-4674-9973-a9bf8da8285b service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:08.571125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:08.571892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:08.572277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:08.572563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] PATH_INFO: `/v3/users/91f74fecab664aeba143108777ee02f0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:08.573780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91f74fecab664aeba143108777ee02f0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:08.575737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:08.604836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aeeb1f-0678-412b-a868-87454d3b2f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:08.606299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aeeb1f-0678-412b-a868-87454d3b2f74 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:08.615804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 938/1853] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:08.779687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91f74fecab664aeba143108777ee02f0'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:08.786465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91f74fecab664aeba143108777ee02f0'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:08.847094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91f74fecab664aeba143108777ee02f0'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:08.847728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91f74fecab664aeba143108777ee02f0'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:08.860174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91f74fecab664aeba143108777ee02f0'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:08.866180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 916/1854] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:08 2022] DELETE /identity/v3/users/91f74fecab664aeba143108777ee02f0 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:08.870540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5656-ffee-4ded-b084-0ea5790ca1ba None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:08.870829 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5656-ffee-4ded-b084-0ea5790ca1ba None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:08.871108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5656-ffee-4ded-b084-0ea5790ca1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:08.892266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4f6a5656-ffee-4ded-b084-0ea5790ca1ba None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:08.893500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4f6a5656-ffee-4ded-b084-0ea5790ca1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:08.953389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 939/1855] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:08.963157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:08.964738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:09.032561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7331533-f6a7-4554-879f-b8473a8c9c01 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:09.042980 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57bcc3-a1ba-49c1-9f0e-f9c7a38f89c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:09.048308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57bcc3-a1ba-49c1-9f0e-f9c7a38f89c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:09.048877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57bcc3-a1ba-49c1-9f0e-f9c7a38f89c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:09.049204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57bcc3-a1ba-49c1-9f0e-f9c7a38f89c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:09.065448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57bcc3-a1ba-49c1-9f0e-f9c7a38f89c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:09.066893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57bcc3-a1ba-49c1-9f0e-f9c7a38f89c8 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:09.077946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 917/1856] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:09.643175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5656-ffee-4ded-b084-0ea5790ca1ba None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:09.644997 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4f6a5656-ffee-4ded-b084-0ea5790ca1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:09.656553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:09.660361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5656-ffee-4ded-b084-0ea5790ca1ba None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:09.666859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:09.677384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:09.677776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:09.678067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] PATH_INFO: `/v3/projects/c1090bdab94a46a1b5dcf823d9b1c4fc` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:09.683543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1090bdab94a46a1b5dcf823d9b1c4fc)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:09.685576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:09.822864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1090bdab94a46a1b5dcf823d9b1c4fc', 'request_id': 'req-98b4b134-48f0-477f-8f3d-e79436338e00'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:09.828294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1090bdab94a46a1b5dcf823d9b1c4fc', 'request_id': 'req-98b4b134-48f0-477f-8f3d-e79436338e00'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:09.834577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1090bdab94a46a1b5dcf823d9b1c4fc', 'request_id': 'req-98b4b134-48f0-477f-8f3d-e79436338e00'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:09.930974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] The token cache is being invalidate because project c1090bdab94a46a1b5dcf823d9b1c4fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:09.931264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:09.931883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:09.933530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 940/1857] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:09 2022] DELETE /identity/v3/projects/c1090bdab94a46a1b5dcf823d9b1c4fc => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:09.937490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57bcc3-a1ba-49c1-9f0e-f9c7a38f89c8 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:09.947021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57bcc3-a1ba-49c1-9f0e-f9c7a38f89c8 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:10.003618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c012aae-7897-4566-98b0-0a8c68420ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:10.004302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c012aae-7897-4566-98b0-0a8c68420ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:10.004588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c012aae-7897-4566-98b0-0a8c68420ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:10.004871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c012aae-7897-4566-98b0-0a8c68420ce7 admin admin] PATH_INFO: `/v3/users/606f4471e9834e57ad1d21cbc8859236` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:10.006288 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c012aae-7897-4566-98b0-0a8c68420ce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=606f4471e9834e57ad1d21cbc8859236)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:10.008222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c012aae-7897-4566-98b0-0a8c68420ce7 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:10.020611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:10.180933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1c012aae-7897-4566-98b0-0a8c68420ce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '606f4471e9834e57ad1d21cbc8859236'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:10.187857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1c012aae-7897-4566-98b0-0a8c68420ce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '606f4471e9834e57ad1d21cbc8859236'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:10.265211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1c012aae-7897-4566-98b0-0a8c68420ce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '606f4471e9834e57ad1d21cbc8859236'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:10.265851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1c012aae-7897-4566-98b0-0a8c68420ce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '606f4471e9834e57ad1d21cbc8859236'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:10.278350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1c012aae-7897-4566-98b0-0a8c68420ce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '606f4471e9834e57ad1d21cbc8859236'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:10.285056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 918/1858] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:09 2022] DELETE /identity/v3/users/606f4471e9834e57ad1d21cbc8859236 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:10.802637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:10.804245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:10.817844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4b134-48f0-477f-8f3d-e79436338e00 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:10.894447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:10.895124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:10.895401 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:10.895937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] PATH_INFO: `/v3/projects/d6343f8e45a849f08eb6e98e2f35203b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:10.897338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6343f8e45a849f08eb6e98e2f35203b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:10.899247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:10.932822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:11.013565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6343f8e45a849f08eb6e98e2f35203b', 'request_id': 'req-fc56fec4-1038-4b23-a637-9e95d2b22de2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:11.018969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6343f8e45a849f08eb6e98e2f35203b', 'request_id': 'req-fc56fec4-1038-4b23-a637-9e95d2b22de2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:11.025172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6343f8e45a849f08eb6e98e2f35203b', 'request_id': 'req-fc56fec4-1038-4b23-a637-9e95d2b22de2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:11.109885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] The token cache is being invalidate because project d6343f8e45a849f08eb6e98e2f35203b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:11.110181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:11.110814 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fc56fec4-1038-4b23-a637-9e95d2b22de2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:11.112338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 941/1859] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:10 2022] DELETE /identity/v3/projects/d6343f8e45a849f08eb6e98e2f35203b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:11.117296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f7fb-bc5a-4912-9727-ca177a2f1ada None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:11.117768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f7fb-bc5a-4912-9727-ca177a2f1ada None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:11.117905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f7fb-bc5a-4912-9727-ca177a2f1ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:11.142399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5582f7fb-bc5a-4912-9727-ca177a2f1ada None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:11.142399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5582f7fb-bc5a-4912-9727-ca177a2f1ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:11.198445 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 919/1860] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:11.206030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d9e96-4794-4ffb-840a-4c9a74774a23 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:11.206333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d9e96-4794-4ffb-840a-4c9a74774a23 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:11.206592 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d9e96-4794-4ffb-840a-4c9a74774a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:11.229547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-dd6d9e96-4794-4ffb-840a-4c9a74774a23 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:11.230784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dd6d9e96-4794-4ffb-840a-4c9a74774a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:11.239592 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 942/1861] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:11.243920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582f7fb-bc5a-4912-9727-ca177a2f1ada None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:11.253779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582f7fb-bc5a-4912-9727-ca177a2f1ada None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:11.258715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812333e-c6f6-4fbf-9582-3600b8ac3eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:11.259461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812333e-c6f6-4fbf-9582-3600b8ac3eae admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:11.259750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812333e-c6f6-4fbf-9582-3600b8ac3eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:11.260032 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812333e-c6f6-4fbf-9582-3600b8ac3eae admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:11.260747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812333e-c6f6-4fbf-9582-3600b8ac3eae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:13:11.261093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812333e-c6f6-4fbf-9582-3600b8ac3eae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:11.262882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812333e-c6f6-4fbf-9582-3600b8ac3eae admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:11.271610 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:11.287014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 920/1862] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:13:11.287115 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d9e96-4794-4ffb-840a-4c9a74774a23 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:11.291038 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d9e96-4794-4ffb-840a-4c9a74774a23 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:11.296376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f3b65-7e3d-4095-a8a7-ba67c46147e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:11.297014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f3b65-7e3d-4095-a8a7-ba67c46147e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:11.297305 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f3b65-7e3d-4095-a8a7-ba67c46147e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:11.297607 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f3b65-7e3d-4095-a8a7-ba67c46147e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:11.298278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f3b65-7e3d-4095-a8a7-ba67c46147e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:11.300164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f3b65-7e3d-4095-a8a7-ba67c46147e7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:11.385516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 943/1863] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:11 2022] POST /identity/v3/projects => generated 391 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:11.390810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812333e-c6f6-4fbf-9582-3600b8ac3eae admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:11.401231 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812333e-c6f6-4fbf-9582-3600b8ac3eae admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:11.465800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ed4ea-6cb9-4279-b692-2565aef3b7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:11.466996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ed4ea-6cb9-4279-b692-2565aef3b7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:11.467312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ed4ea-6cb9-4279-b692-2565aef3b7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:11.467590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ed4ea-6cb9-4279-b692-2565aef3b7d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:11.468265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ed4ea-6cb9-4279-b692-2565aef3b7d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:11.470488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ed4ea-6cb9-4279-b692-2565aef3b7d7 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:11.544361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 921/1864] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:11 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:11.548382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f3b65-7e3d-4095-a8a7-ba67c46147e7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:11.558681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f3b65-7e3d-4095-a8a7-ba67c46147e7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:11.563288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46df0ad-eb5c-4572-925e-2bd758776c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:11.563899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46df0ad-eb5c-4572-925e-2bd758776c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:11.564182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46df0ad-eb5c-4572-925e-2bd758776c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:11.564458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46df0ad-eb5c-4572-925e-2bd758776c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:11.565219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46df0ad-eb5c-4572-925e-2bd758776c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:11.583795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46df0ad-eb5c-4572-925e-2bd758776c37 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:11.583795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 944/1865] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:11.583983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ed4ea-6cb9-4279-b692-2565aef3b7d7 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:11.592485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ed4ea-6cb9-4279-b692-2565aef3b7d7 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:11.598098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6abae-06d1-4db2-80f1-7184874be0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:11.598709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6abae-06d1-4db2-80f1-7184874be0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:11.598993 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6abae-06d1-4db2-80f1-7184874be0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:11.599270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6abae-06d1-4db2-80f1-7184874be0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:11.600009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6abae-06d1-4db2-80f1-7184874be0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:11.601678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6abae-06d1-4db2-80f1-7184874be0b3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:11.612209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 922/1866] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:13:11.616146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46df0ad-eb5c-4572-925e-2bd758776c37 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:11.626849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46df0ad-eb5c-4572-925e-2bd758776c37 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:11.631617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748c1f8-7db2-41f8-b3c2-6818e429e060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:11.632278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748c1f8-7db2-41f8-b3c2-6818e429e060 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:11.633756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748c1f8-7db2-41f8-b3c2-6818e429e060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:11.633756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748c1f8-7db2-41f8-b3c2-6818e429e060 admin admin] PATH_INFO: `/v3/projects/b30f71b45b0c45ad82e4771b3541713c/users/84a2c2eb87cd4566bacb5847b4e1be88/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:11.649641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748c1f8-7db2-41f8-b3c2-6818e429e060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30f71b45b0c45ad82e4771b3541713c, user_id=84a2c2eb87cd4566bacb5847b4e1be88, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:11.652772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748c1f8-7db2-41f8-b3c2-6818e429e060 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:11.664528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:11.699664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 945/1867] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:11 2022] PUT /identity/v3/projects/b30f71b45b0c45ad82e4771b3541713c/users/84a2c2eb87cd4566bacb5847b4e1be88/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:11.710837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718b84a-3da4-4ce4-b25b-7c003db15930 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:11.711132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718b84a-3da4-4ce4-b25b-7c003db15930 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:11.711415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718b84a-3da4-4ce4-b25b-7c003db15930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:11.740553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6718b84a-3da4-4ce4-b25b-7c003db15930 None None] MFA Rules not processed for user `84a2c2eb87cd4566bacb5847b4e1be88`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:11.741823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6718b84a-3da4-4ce4-b25b-7c003db15930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:11.818401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 923/1868] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:11.823037 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748c1f8-7db2-41f8-b3c2-6818e429e060 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:11.889911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748c1f8-7db2-41f8-b3c2-6818e429e060 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:11.896597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5fdb8-aae8-4246-a8bd-9559dafb5cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:11.897263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5fdb8-aae8-4246-a8bd-9559dafb5cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:11.897546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5fdb8-aae8-4246-a8bd-9559dafb5cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:11.897828 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5fdb8-aae8-4246-a8bd-9559dafb5cd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:11.898499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5fdb8-aae8-4246-a8bd-9559dafb5cd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:11.900421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5fdb8-aae8-4246-a8bd-9559dafb5cd3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:12.000593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:12.011535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 946/1869] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:11 2022] POST /identity/v3/projects => generated 391 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:13:12.012748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718b84a-3da4-4ce4-b25b-7c003db15930 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:12.024799 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718b84a-3da4-4ce4-b25b-7c003db15930 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:12.088754 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345fd60-a6bf-492c-bcbd-a862ef6a72dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:12.089765 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345fd60-a6bf-492c-bcbd-a862ef6a72dd admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:12.090185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345fd60-a6bf-492c-bcbd-a862ef6a72dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:12.090584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345fd60-a6bf-492c-bcbd-a862ef6a72dd admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:12.091587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345fd60-a6bf-492c-bcbd-a862ef6a72dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:12.094898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345fd60-a6bf-492c-bcbd-a862ef6a72dd admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:12.198687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 924/1870] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:12 2022] POST /identity/v3/users => generated 302 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:12.199924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5fdb8-aae8-4246-a8bd-9559dafb5cd3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:12.211100 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5fdb8-aae8-4246-a8bd-9559dafb5cd3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:12.216928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c867e8b-c3c9-447f-8af3-ee1684a9e58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:12.217653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c867e8b-c3c9-447f-8af3-ee1684a9e58b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:12.217942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c867e8b-c3c9-447f-8af3-ee1684a9e58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:12.218217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c867e8b-c3c9-447f-8af3-ee1684a9e58b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:12.221798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c867e8b-c3c9-447f-8af3-ee1684a9e58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:12.221798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c867e8b-c3c9-447f-8af3-ee1684a9e58b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:12.231841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 947/1871] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:13:12.236219 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345fd60-a6bf-492c-bcbd-a862ef6a72dd admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:12.246533 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345fd60-a6bf-492c-bcbd-a862ef6a72dd admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:12.251254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4656f-fef0-4bfa-a59d-4ba6b39c7d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:12.251922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4656f-fef0-4bfa-a59d-4ba6b39c7d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:12.252206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4656f-fef0-4bfa-a59d-4ba6b39c7d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:12.252483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4656f-fef0-4bfa-a59d-4ba6b39c7d02 admin admin] PATH_INFO: `/v3/projects/e89b0a3c215f439d9358cd07575c9b4f/users/49a571309b574187b954c752a270d8cb/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:12.268152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4656f-fef0-4bfa-a59d-4ba6b39c7d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89b0a3c215f439d9358cd07575c9b4f, user_id=49a571309b574187b954c752a270d8cb, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:12.271028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4656f-fef0-4bfa-a59d-4ba6b39c7d02 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:12.371238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 925/1872] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:12 2022] PUT /identity/v3/projects/e89b0a3c215f439d9358cd07575c9b4f/users/49a571309b574187b954c752a270d8cb/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:12.372276 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c867e8b-c3c9-447f-8af3-ee1684a9e58b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:12.382677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c867e8b-c3c9-447f-8af3-ee1684a9e58b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:12.439694 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384bf41-0330-48fb-9e1f-785731e83a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:12.440347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384bf41-0330-48fb-9e1f-785731e83a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:12.440633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384bf41-0330-48fb-9e1f-785731e83a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:12.440911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384bf41-0330-48fb-9e1f-785731e83a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:12.441640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384bf41-0330-48fb-9e1f-785731e83a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:12.443221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384bf41-0330-48fb-9e1f-785731e83a64 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:12.453073 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 948/1873] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:12.457318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4656f-fef0-4bfa-a59d-4ba6b39c7d02 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:12.468129 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4656f-fef0-4bfa-a59d-4ba6b39c7d02 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:12.473176 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2e330-fc7f-4190-a2a6-ac408d2c7888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:12.473845 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2e330-fc7f-4190-a2a6-ac408d2c7888 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:12.474133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2e330-fc7f-4190-a2a6-ac408d2c7888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:12.474463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2e330-fc7f-4190-a2a6-ac408d2c7888 admin admin] PATH_INFO: `/v3/projects/e89b0a3c215f439d9358cd07575c9b4f/users/49a571309b574187b954c752a270d8cb/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:12.476421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2e330-fc7f-4190-a2a6-ac408d2c7888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89b0a3c215f439d9358cd07575c9b4f, user_id=49a571309b574187b954c752a270d8cb, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:12.479094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2e330-fc7f-4190-a2a6-ac408d2c7888 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:12.489907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 926/1874] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:12 2022] PUT /identity/v3/projects/e89b0a3c215f439d9358cd07575c9b4f/users/49a571309b574187b954c752a270d8cb/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:12.501868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e1609-d2f0-4f4f-947b-bc02be122210 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:12.502165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e1609-d2f0-4f4f-947b-bc02be122210 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:12.502483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e1609-d2f0-4f4f-947b-bc02be122210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:12.532372 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9e9e1609-d2f0-4f4f-947b-bc02be122210 None None] MFA Rules not processed for user `49a571309b574187b954c752a270d8cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:12.533649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9e9e1609-d2f0-4f4f-947b-bc02be122210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:12.614512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 949/1875] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:12.805149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2e330-fc7f-4190-a2a6-ac408d2c7888 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:12.806849 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-27e2e330-fc7f-4190-a2a6-ac408d2c7888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:12.830573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:12.866842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2e330-fc7f-4190-a2a6-ac408d2c7888 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:12.870879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e425d77-26ec-4a9e-8923-17e250f6a194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:12.871523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e425d77-26ec-4a9e-8923-17e250f6a194 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:12.871803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e425d77-26ec-4a9e-8923-17e250f6a194 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:12.872071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e425d77-26ec-4a9e-8923-17e250f6a194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:12.881600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e425d77-26ec-4a9e-8923-17e250f6a194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:12.883103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e425d77-26ec-4a9e-8923-17e250f6a194 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:12.893245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 927/1876] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:13.011477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e1609-d2f0-4f4f-947b-bc02be122210 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:13.017989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9e9e1609-d2f0-4f4f-947b-bc02be122210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:13.031632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e1609-d2f0-4f4f-947b-bc02be122210 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:13.036780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9845b25-4985-4fc4-afaf-a1b6f29fe3d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:13.037600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9845b25-4985-4fc4-afaf-a1b6f29fe3d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:13.037885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9845b25-4985-4fc4-afaf-a1b6f29fe3d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:13.038150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9845b25-4985-4fc4-afaf-a1b6f29fe3d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:13.109076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9845b25-4985-4fc4-afaf-a1b6f29fe3d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:13.110974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9845b25-4985-4fc4-afaf-a1b6f29fe3d8 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:13.130233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 950/1877] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:13.807002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:14.383694 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:14.561062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:14.669877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:14.937126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:14.959921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:15.005527 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:15.074400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:15.097171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:15.189814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:15.217573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:15.285665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:15.359849 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:15.481659 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:17.544740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf62b1-5c64-4e81-81d3-14cfb962d535 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:17.546011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf62b1-5c64-4e81-81d3-14cfb962d535 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:17.546011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf62b1-5c64-4e81-81d3-14cfb962d535 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:17.547339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 928/1878] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:13:17.554170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e279f-318d-4aef-a1e1-e7ed3bc968f9 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:17.554603 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e279f-318d-4aef-a1e1-e7ed3bc968f9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:17.554893 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e279f-318d-4aef-a1e1-e7ed3bc968f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:17.574955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-876e279f-318d-4aef-a1e1-e7ed3bc968f9 None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:17.576189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-876e279f-318d-4aef-a1e1-e7ed3bc968f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:17.645166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 951/1879] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:13:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:17.660742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf62b1-5c64-4e81-81d3-14cfb962d535 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:17.662515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4abf62b1-5c64-4e81-81d3-14cfb962d535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:17.683309 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf62b1-5c64-4e81-81d3-14cfb962d535 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:17.689819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9479d7-b26d-4adf-8808-fdff08bd01db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:17.690446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9479d7-b26d-4adf-8808-fdff08bd01db service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:17.690728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9479d7-b26d-4adf-8808-fdff08bd01db service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:17.691004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9479d7-b26d-4adf-8808-fdff08bd01db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:17.697126 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:17.701065 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9479d7-b26d-4adf-8808-fdff08bd01db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:17.702714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9479d7-b26d-4adf-8808-fdff08bd01db service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:17.713356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 929/1880] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:26.929109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e279f-318d-4aef-a1e1-e7ed3bc968f9 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:26.998414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e279f-318d-4aef-a1e1-e7ed3bc968f9 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:27.004822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:27.005701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:27.005986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:27.006269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] PATH_INFO: `/v3/users/84a2c2eb87cd4566bacb5847b4e1be88` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:27.011535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] RBAC: Authorizing `identity:delete_user(user_id=84a2c2eb87cd4566bacb5847b4e1be88)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:27.011535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:27.230321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84a2c2eb87cd4566bacb5847b4e1be88'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:27.233337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84a2c2eb87cd4566bacb5847b4e1be88'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:27.242869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84a2c2eb87cd4566bacb5847b4e1be88'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:27.332091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84a2c2eb87cd4566bacb5847b4e1be88'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:27.333055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84a2c2eb87cd4566bacb5847b4e1be88'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:27.341251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:27.354516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 952/1881] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:26 2022] DELETE /identity/v3/users/84a2c2eb87cd4566bacb5847b4e1be88 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:27.359359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750572f-b8fe-41b7-8038-7c2e866e8018 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:27.360343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750572f-b8fe-41b7-8038-7c2e866e8018 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:27.360774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750572f-b8fe-41b7-8038-7c2e866e8018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:27.385555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-4750572f-b8fe-41b7-8038-7c2e866e8018 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:27.386909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4750572f-b8fe-41b7-8038-7c2e866e8018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:27.527473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 930/1882] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:13:27.537130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:27.538634 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:27.631951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569f790-e575-43e9-8255-90782f35d53a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:27.638934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4ff6c-bc51-4046-a795-6074dbd997c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:27.639322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4ff6c-bc51-4046-a795-6074dbd997c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:27.639986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4ff6c-bc51-4046-a795-6074dbd997c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:27.639986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4ff6c-bc51-4046-a795-6074dbd997c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:27.651514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4ff6c-bc51-4046-a795-6074dbd997c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:27.653113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4ff6c-bc51-4046-a795-6074dbd997c8 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:27.663978 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 953/1883] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:28.683713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750572f-b8fe-41b7-8038-7c2e866e8018 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:28.686218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4750572f-b8fe-41b7-8038-7c2e866e8018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:28.722383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750572f-b8fe-41b7-8038-7c2e866e8018 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:28.730650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:28.731455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:28.731699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:28.732381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] PATH_INFO: `/v3/projects/b30f71b45b0c45ad82e4771b3541713c` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:28.734438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30f71b45b0c45ad82e4771b3541713c)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:28.740149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:28.906200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30f71b45b0c45ad82e4771b3541713c', 'request_id': 'req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:28.918725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30f71b45b0c45ad82e4771b3541713c', 'request_id': 'req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:29.002642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30f71b45b0c45ad82e4771b3541713c', 'request_id': 'req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:29.020623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] The token cache is being invalidate because project b30f71b45b0c45ad82e4771b3541713c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:29.020623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:29.021432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:29.024452 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 931/1884] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:28 2022] DELETE /identity/v3/projects/b30f71b45b0c45ad82e4771b3541713c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:29.028186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4ff6c-bc51-4046-a795-6074dbd997c8 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:29.040286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4ff6c-bc51-4046-a795-6074dbd997c8 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:29.124985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa18c-05b0-4d17-b66a-34762ced8148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:29.125701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa18c-05b0-4d17-b66a-34762ced8148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:29.125988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa18c-05b0-4d17-b66a-34762ced8148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:29.126270 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa18c-05b0-4d17-b66a-34762ced8148 admin admin] PATH_INFO: `/v3/users/49a571309b574187b954c752a270d8cb` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:29.129596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fa18c-05b0-4d17-b66a-34762ced8148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a571309b574187b954c752a270d8cb)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:29.131575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fa18c-05b0-4d17-b66a-34762ced8148 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:29.463016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-889fa18c-05b0-4d17-b66a-34762ced8148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a571309b574187b954c752a270d8cb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:29.466688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-889fa18c-05b0-4d17-b66a-34762ced8148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a571309b574187b954c752a270d8cb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:29.486391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-889fa18c-05b0-4d17-b66a-34762ced8148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a571309b574187b954c752a270d8cb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:29.570927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-889fa18c-05b0-4d17-b66a-34762ced8148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a571309b574187b954c752a270d8cb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:29.572601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-889fa18c-05b0-4d17-b66a-34762ced8148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a571309b574187b954c752a270d8cb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:29.601153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 954/1885] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:29 2022] DELETE /identity/v3/users/49a571309b574187b954c752a270d8cb => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:30.043657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:30.045940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:30.064119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6cde0-ad10-42e1-a17e-0ba0fbfd9288 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:30.135929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:30.136655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:30.136946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:30.137257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] PATH_INFO: `/v3/projects/e89b0a3c215f439d9358cd07575c9b4f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:30.138457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e89b0a3c215f439d9358cd07575c9b4f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:30.140432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:30.164510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:30.248209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e89b0a3c215f439d9358cd07575c9b4f', 'request_id': 'req-0f54505f-acf0-4b40-9bcb-739070b39ddf'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:30.254859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e89b0a3c215f439d9358cd07575c9b4f', 'request_id': 'req-0f54505f-acf0-4b40-9bcb-739070b39ddf'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:30.348631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e89b0a3c215f439d9358cd07575c9b4f', 'request_id': 'req-0f54505f-acf0-4b40-9bcb-739070b39ddf'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:30.360153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] The token cache is being invalidate because project e89b0a3c215f439d9358cd07575c9b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:30.360508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:30.361261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0f54505f-acf0-4b40-9bcb-739070b39ddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:30.362826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 932/1886] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:30 2022] DELETE /identity/v3/projects/e89b0a3c215f439d9358cd07575c9b4f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:30.367770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef5e0c-07bc-42a0-99cb-bc12df3f1821 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:30.368247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef5e0c-07bc-42a0-99cb-bc12df3f1821 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:30.368396 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef5e0c-07bc-42a0-99cb-bc12df3f1821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:30.388536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-bbef5e0c-07bc-42a0-99cb-bc12df3f1821 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:30.389799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bbef5e0c-07bc-42a0-99cb-bc12df3f1821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:30.394929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:30.451891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 955/1887] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:30.460532 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff8f39-159b-4816-be28-cb2546803e4e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:30.460824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff8f39-159b-4816-be28-cb2546803e4e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:30.461148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff8f39-159b-4816-be28-cb2546803e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:30.481407 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-48ff8f39-159b-4816-be28-cb2546803e4e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:30.482652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-48ff8f39-159b-4816-be28-cb2546803e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:30.490977 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 933/1888] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:30.495059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef5e0c-07bc-42a0-99cb-bc12df3f1821 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:30.506114 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef5e0c-07bc-42a0-99cb-bc12df3f1821 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:30.511444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6c08c-8d0a-4057-932b-022572577e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:30.512151 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c08c-8d0a-4057-932b-022572577e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:30.512443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c08c-8d0a-4057-932b-022572577e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:30.513052 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c08c-8d0a-4057-932b-022572577e3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:30.514126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6c08c-8d0a-4057-932b-022572577e3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:13:30.514126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6c08c-8d0a-4057-932b-022572577e3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:30.515743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6c08c-8d0a-4057-932b-022572577e3b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:30.535485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 956/1889] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:13:30.539674 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff8f39-159b-4816-be28-cb2546803e4e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:30.550596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff8f39-159b-4816-be28-cb2546803e4e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:30.558047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4bda9-4a10-443c-9881-2d33dc6afe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:30.558720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4bda9-4a10-443c-9881-2d33dc6afe6a admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:30.559021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4bda9-4a10-443c-9881-2d33dc6afe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:30.559806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4bda9-4a10-443c-9881-2d33dc6afe6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:30.560034 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4bda9-4a10-443c-9881-2d33dc6afe6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:30.562037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4bda9-4a10-443c-9881-2d33dc6afe6a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:30.652182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 934/1890] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:30 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:13:30.653247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6c08c-8d0a-4057-932b-022572577e3b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:30.663499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6c08c-8d0a-4057-932b-022572577e3b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:30.723554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09969-cb29-4531-948f-75672002e217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:30.724221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09969-cb29-4531-948f-75672002e217 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:30.724613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09969-cb29-4531-948f-75672002e217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:30.724853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09969-cb29-4531-948f-75672002e217 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:30.725577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe09969-cb29-4531-948f-75672002e217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:30.727507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe09969-cb29-4531-948f-75672002e217 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:30.829620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 957/1891] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:30 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:30.833957 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4bda9-4a10-443c-9881-2d33dc6afe6a admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:30.844769 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4bda9-4a10-443c-9881-2d33dc6afe6a admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:30.849410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8b757-2c80-4c88-abe4-caf5f965e05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:30.850009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8b757-2c80-4c88-abe4-caf5f965e05c admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:30.850292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8b757-2c80-4c88-abe4-caf5f965e05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:30.850567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8b757-2c80-4c88-abe4-caf5f965e05c admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:30.851296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8b757-2c80-4c88-abe4-caf5f965e05c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:30.852923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8b757-2c80-4c88-abe4-caf5f965e05c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:30.863478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 935/1892] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:30.867556 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09969-cb29-4531-948f-75672002e217 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:30.878587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09969-cb29-4531-948f-75672002e217 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:30.883204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe90c4-3354-4a3c-9a5f-f72386bce391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:30.883836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe90c4-3354-4a3c-9a5f-f72386bce391 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:30.884109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe90c4-3354-4a3c-9a5f-f72386bce391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:30.884392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe90c4-3354-4a3c-9a5f-f72386bce391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:30.885155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe90c4-3354-4a3c-9a5f-f72386bce391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:30.886764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe90c4-3354-4a3c-9a5f-f72386bce391 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:30.897910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 958/1893] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:30.902069 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8b757-2c80-4c88-abe4-caf5f965e05c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:30.912106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8b757-2c80-4c88-abe4-caf5f965e05c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:30.916702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8822ac7-6f34-4fe5-b86b-7160e257ba9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:30.917385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8822ac7-6f34-4fe5-b86b-7160e257ba9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:30.917668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8822ac7-6f34-4fe5-b86b-7160e257ba9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:30.917942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8822ac7-6f34-4fe5-b86b-7160e257ba9b admin admin] PATH_INFO: `/v3/projects/9cfde5a784a84d9bb04353268470219e/users/66c90572f6bb47c39f2855933b69b37b/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:30.933705 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8822ac7-6f34-4fe5-b86b-7160e257ba9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cfde5a784a84d9bb04353268470219e, user_id=66c90572f6bb47c39f2855933b69b37b, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:30.936336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8822ac7-6f34-4fe5-b86b-7160e257ba9b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:31.005094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 936/1894] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:30 2022] PUT /identity/v3/projects/9cfde5a784a84d9bb04353268470219e/users/66c90572f6bb47c39f2855933b69b37b/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:31.016244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2168ca-e6e1-4e10-b99e-bc733ef6f36e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:31.016535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2168ca-e6e1-4e10-b99e-bc733ef6f36e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:31.016815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2168ca-e6e1-4e10-b99e-bc733ef6f36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:31.045700 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7b2168ca-e6e1-4e10-b99e-bc733ef6f36e None None] MFA Rules not processed for user `66c90572f6bb47c39f2855933b69b37b`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:31.046919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7b2168ca-e6e1-4e10-b99e-bc733ef6f36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:31.113939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 959/1895] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:31.124727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8822ac7-6f34-4fe5-b86b-7160e257ba9b admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:31.126603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a8822ac7-6f34-4fe5-b86b-7160e257ba9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:31.213323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8822ac7-6f34-4fe5-b86b-7160e257ba9b admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:31.217317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206c86-328f-4e23-ab8d-e3800ebbad99 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47f1552e73648ceb8a421c0543c2974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:31.218194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206c86-328f-4e23-ab8d-e3800ebbad99 service glance] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:31.218491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206c86-328f-4e23-ab8d-e3800ebbad99 service glance] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:31.218755 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206c86-328f-4e23-ab8d-e3800ebbad99 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:31.227510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17206c86-328f-4e23-ab8d-e3800ebbad99 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:31.228984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17206c86-328f-4e23-ab8d-e3800ebbad99 service glance] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:31.239484 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 937/1896] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:41.110682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:41.235261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:47.610718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2168ca-e6e1-4e10-b99e-bc733ef6f36e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:47.612133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7b2168ca-e6e1-4e10-b99e-bc733ef6f36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:47.697058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2168ca-e6e1-4e10-b99e-bc733ef6f36e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:47.701989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:47.702648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:47.702953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:47.703266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] PATH_INFO: `/v3/users/f93cfa81261744259dc1075307bd9ca4` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:47.704453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f93cfa81261744259dc1075307bd9ca4)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:47.706411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:47.915497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f93cfa81261744259dc1075307bd9ca4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:47.916919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f93cfa81261744259dc1075307bd9ca4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:47.936136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f93cfa81261744259dc1075307bd9ca4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:48.057730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f93cfa81261744259dc1075307bd9ca4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:48.070521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f93cfa81261744259dc1075307bd9ca4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:48.086091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 960/1897] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:47 2022] DELETE /identity/v3/users/f93cfa81261744259dc1075307bd9ca4 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:48.091700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b07b78-4a6b-40ac-afe2-0c3912d02582 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:48.092453 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b07b78-4a6b-40ac-afe2-0c3912d02582 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:48.093137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b07b78-4a6b-40ac-afe2-0c3912d02582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:48.115007 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-25b07b78-4a6b-40ac-afe2-0c3912d02582 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:48.117510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-25b07b78-4a6b-40ac-afe2-0c3912d02582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:48.186345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 938/1898] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:48.197423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:48.198849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:48.255682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf3b8c-6ee3-4ef7-acca-5de3402d5766 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:48.260058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94772e-7418-4019-aa49-0ff7b58e63e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:48.260720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94772e-7418-4019-aa49-0ff7b58e63e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:48.261088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94772e-7418-4019-aa49-0ff7b58e63e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:48.261375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94772e-7418-4019-aa49-0ff7b58e63e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:48.272007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94772e-7418-4019-aa49-0ff7b58e63e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:48.273527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94772e-7418-4019-aa49-0ff7b58e63e0 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:48.283845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 961/1899] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:48.761670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b07b78-4a6b-40ac-afe2-0c3912d02582 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:48.763254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-25b07b78-4a6b-40ac-afe2-0c3912d02582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:48.778633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b07b78-4a6b-40ac-afe2-0c3912d02582 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:48.783449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:48.784848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:48.785196 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:48.785859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] PATH_INFO: `/v3/projects/57de1e0db9be4cc2ac06c5636fe44c2e` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:48.788448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57de1e0db9be4cc2ac06c5636fe44c2e)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:48.790067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:48.925542 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57de1e0db9be4cc2ac06c5636fe44c2e', 'request_id': 'req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:48.932523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57de1e0db9be4cc2ac06c5636fe44c2e', 'request_id': 'req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:48.994293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57de1e0db9be4cc2ac06c5636fe44c2e', 'request_id': 'req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:49.006979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] The token cache is being invalidate because project 57de1e0db9be4cc2ac06c5636fe44c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:49.007272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:49.007918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:49.009500 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 939/1900] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:48 2022] DELETE /identity/v3/projects/57de1e0db9be4cc2ac06c5636fe44c2e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:49.013450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94772e-7418-4019-aa49-0ff7b58e63e0 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:49.023198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94772e-7418-4019-aa49-0ff7b58e63e0 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:49.042135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:49.080894 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37014e85-f6e3-4a15-96b6-4a96f0ddc063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:49.081666 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37014e85-f6e3-4a15-96b6-4a96f0ddc063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:49.081948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37014e85-f6e3-4a15-96b6-4a96f0ddc063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:49.082222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37014e85-f6e3-4a15-96b6-4a96f0ddc063 admin admin] PATH_INFO: `/v3/users/ceea33185b884d1aa503b479b6d530e8` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:49.083496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37014e85-f6e3-4a15-96b6-4a96f0ddc063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceea33185b884d1aa503b479b6d530e8)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:49.086013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37014e85-f6e3-4a15-96b6-4a96f0ddc063 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:49.357873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-37014e85-f6e3-4a15-96b6-4a96f0ddc063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceea33185b884d1aa503b479b6d530e8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:49.358553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-37014e85-f6e3-4a15-96b6-4a96f0ddc063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceea33185b884d1aa503b479b6d530e8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:49.366512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-37014e85-f6e3-4a15-96b6-4a96f0ddc063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceea33185b884d1aa503b479b6d530e8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:49.446067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-37014e85-f6e3-4a15-96b6-4a96f0ddc063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceea33185b884d1aa503b479b6d530e8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:49.449665 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-37014e85-f6e3-4a15-96b6-4a96f0ddc063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceea33185b884d1aa503b479b6d530e8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:49.476258 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 962/1901] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:49 2022] DELETE /identity/v3/users/ceea33185b884d1aa503b479b6d530e8 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:50.045962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:50.048146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:50.079691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658a7fc-c9ac-48b0-b0f7-5ebfb9894f67 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:50.152766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:50.157621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:50.157621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:50.157621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] PATH_INFO: `/v3/projects/3ff6e6037d874d9d817bf13b432c98d8` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:50.157621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ff6e6037d874d9d817bf13b432c98d8)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:50.157621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:50.258804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:50.275310 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ff6e6037d874d9d817bf13b432c98d8', 'request_id': 'req-e9b43755-1de9-4d39-b726-b705924505c3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:50.285310 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ff6e6037d874d9d817bf13b432c98d8', 'request_id': 'req-e9b43755-1de9-4d39-b726-b705924505c3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:50.405767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ff6e6037d874d9d817bf13b432c98d8', 'request_id': 'req-e9b43755-1de9-4d39-b726-b705924505c3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:50.435742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] The token cache is being invalidate because project 3ff6e6037d874d9d817bf13b432c98d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:50.436150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:50.441599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e9b43755-1de9-4d39-b726-b705924505c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:50.443293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 940/1902] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:50 2022] DELETE /identity/v3/projects/3ff6e6037d874d9d817bf13b432c98d8 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:50.448474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b659d58-51b2-42cb-ae90-57429bba5a4f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:50.448992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b659d58-51b2-42cb-ae90-57429bba5a4f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:50.449089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b659d58-51b2-42cb-ae90-57429bba5a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:50.487154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6b659d58-51b2-42cb-ae90-57429bba5a4f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:50.488597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6b659d58-51b2-42cb-ae90-57429bba5a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:50.579988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 963/1903] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:50.588094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c117d-2414-4145-858c-03c136e15a65 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:50.588240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c117d-2414-4145-858c-03c136e15a65 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:50.588590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c117d-2414-4145-858c-03c136e15a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:50.613202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-023c117d-2414-4145-858c-03c136e15a65 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:50.614745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-023c117d-2414-4145-858c-03c136e15a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:50.626696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 941/1904] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:13:50.630000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b659d58-51b2-42cb-ae90-57429bba5a4f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:50.649242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b659d58-51b2-42cb-ae90-57429bba5a4f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:50.660541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d98ca3-eae0-417c-b577-25f72ca0b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:50.661564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d98ca3-eae0-417c-b577-25f72ca0b2da admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:50.662534 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d98ca3-eae0-417c-b577-25f72ca0b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:50.662975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d98ca3-eae0-417c-b577-25f72ca0b2da admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:50.663974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d98ca3-eae0-417c-b577-25f72ca0b2da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:13:50.664427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d98ca3-eae0-417c-b577-25f72ca0b2da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:50.666364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d98ca3-eae0-417c-b577-25f72ca0b2da admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:50.692197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 964/1905] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:13:50.696512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c117d-2414-4145-858c-03c136e15a65 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:50.707576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c117d-2414-4145-858c-03c136e15a65 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:50.713131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb66b3-5f84-4e8f-859d-4bd5910d28f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:50.713852 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb66b3-5f84-4e8f-859d-4bd5910d28f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:50.714141 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb66b3-5f84-4e8f-859d-4bd5910d28f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:50.714421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb66b3-5f84-4e8f-859d-4bd5910d28f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:50.715103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb66b3-5f84-4e8f-859d-4bd5910d28f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:50.717023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb66b3-5f84-4e8f-859d-4bd5910d28f4 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:50.728492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d98ca3-eae0-417c-b577-25f72ca0b2da admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:50.730483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-53d98ca3-eae0-417c-b577-25f72ca0b2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:50.744393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d98ca3-eae0-417c-b577-25f72ca0b2da admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:50.749199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50893d08-d4c5-458d-a258-bf8c6bd22551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:50.749843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50893d08-d4c5-458d-a258-bf8c6bd22551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:50.750128 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50893d08-d4c5-458d-a258-bf8c6bd22551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:50.750408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50893d08-d4c5-458d-a258-bf8c6bd22551 admin admin] PATH_INFO: `/v3/users/6e9e73d29ed64bad8394004fec02e16d` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:50.751592 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50893d08-d4c5-458d-a258-bf8c6bd22551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9e73d29ed64bad8394004fec02e16d)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:50.753633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50893d08-d4c5-458d-a258-bf8c6bd22551 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:50.776571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:50.839466 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 942/1906] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:50 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:50.840782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb66b3-5f84-4e8f-859d-4bd5910d28f4 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:50.912641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb66b3-5f84-4e8f-859d-4bd5910d28f4 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:50.918096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36043031-ad83-42bf-9652-c947f2c31f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:50.918546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36043031-ad83-42bf-9652-c947f2c31f11 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:50.918821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36043031-ad83-42bf-9652-c947f2c31f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:50.919088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36043031-ad83-42bf-9652-c947f2c31f11 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:50.920358 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36043031-ad83-42bf-9652-c947f2c31f11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:50.921755 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36043031-ad83-42bf-9652-c947f2c31f11 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:50.965407 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-50893d08-d4c5-458d-a258-bf8c6bd22551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9e73d29ed64bad8394004fec02e16d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:50.966110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-50893d08-d4c5-458d-a258-bf8c6bd22551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9e73d29ed64bad8394004fec02e16d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:50.976877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-50893d08-d4c5-458d-a258-bf8c6bd22551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9e73d29ed64bad8394004fec02e16d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:51.015784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 943/1907] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:50 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:51.017324 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36043031-ad83-42bf-9652-c947f2c31f11 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:51.033729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36043031-ad83-42bf-9652-c947f2c31f11 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:51.041046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fdf9b-8666-4aed-8916-b9a0021fe9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:51.041803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fdf9b-8666-4aed-8916-b9a0021fe9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:51.042207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fdf9b-8666-4aed-8916-b9a0021fe9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:51.042553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fdf9b-8666-4aed-8916-b9a0021fe9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:51.043587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fdf9b-8666-4aed-8916-b9a0021fe9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:51.045459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fdf9b-8666-4aed-8916-b9a0021fe9fb admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:51.069005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 944/1908] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:51.069988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-50893d08-d4c5-458d-a258-bf8c6bd22551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9e73d29ed64bad8394004fec02e16d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:51.069988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-50893d08-d4c5-458d-a258-bf8c6bd22551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9e73d29ed64bad8394004fec02e16d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:51.081117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fdf9b-8666-4aed-8916-b9a0021fe9fb admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:51.081117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-158fdf9b-8666-4aed-8916-b9a0021fe9fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:51.090459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 965/1909] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:50 2022] DELETE /identity/v3/users/6e9e73d29ed64bad8394004fec02e16d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:51.091864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fdf9b-8666-4aed-8916-b9a0021fe9fb admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:51.093498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53019784-575a-428e-a887-d0c210e78e4a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:51.093937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53019784-575a-428e-a887-d0c210e78e4a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:51.094465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53019784-575a-428e-a887-d0c210e78e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:51.123362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-53019784-575a-428e-a887-d0c210e78e4a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:51.124884 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-53019784-575a-428e-a887-d0c210e78e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:51.194083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75786df9-f7ed-4261-94f2-0b462c096b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:51.194751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75786df9-f7ed-4261-94f2-0b462c096b5f admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:51.195027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75786df9-f7ed-4261-94f2-0b462c096b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:51.195340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75786df9-f7ed-4261-94f2-0b462c096b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:51.196022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75786df9-f7ed-4261-94f2-0b462c096b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:51.197685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75786df9-f7ed-4261-94f2-0b462c096b5f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:51.221072 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 966/1910] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:51.227308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 945/1911] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:51.229185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:51.232394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53019784-575a-428e-a887-d0c210e78e4a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:51.241003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75786df9-f7ed-4261-94f2-0b462c096b5f admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:51.253844 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-75786df9-f7ed-4261-94f2-0b462c096b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:51.274670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53019784-575a-428e-a887-d0c210e78e4a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:51.280164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a3778-2182-41ae-8fed-ac1274b1be31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:51.280941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3778-2182-41ae-8fed-ac1274b1be31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:51.281273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3778-2182-41ae-8fed-ac1274b1be31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:51.281683 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3778-2182-41ae-8fed-ac1274b1be31 admin admin] PATH_INFO: `/v3/projects/e2b32fcdd0ad41ea9a821bc81a0f1d03/users/29abe018b1dc4da69a55c75be6677fcb/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:51.301685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a3778-2182-41ae-8fed-ac1274b1be31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b32fcdd0ad41ea9a821bc81a0f1d03, user_id=29abe018b1dc4da69a55c75be6677fcb, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:51.305924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a3778-2182-41ae-8fed-ac1274b1be31 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:51.321625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75786df9-f7ed-4261-94f2-0b462c096b5f admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:51.325495 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1bf0a-e207-4eb7-b669-fc6d280ce94d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:51.326150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1bf0a-e207-4eb7-b669-fc6d280ce94d service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:51.326451 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1bf0a-e207-4eb7-b669-fc6d280ce94d service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:51.326733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1bf0a-e207-4eb7-b669-fc6d280ce94d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:51.340505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1bf0a-e207-4eb7-b669-fc6d280ce94d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:51.342042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1bf0a-e207-4eb7-b669-fc6d280ce94d service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:51.358009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 946/1912] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:13:51.390768 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 967/1913] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:51 2022] PUT /identity/v3/projects/e2b32fcdd0ad41ea9a821bc81a0f1d03/users/29abe018b1dc4da69a55c75be6677fcb/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:51.402437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0f08e-e782-4414-a15a-748106aa436b None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:51.402732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0f08e-e782-4414-a15a-748106aa436b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:51.403020 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0f08e-e782-4414-a15a-748106aa436b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:51.440028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-b6d0f08e-e782-4414-a15a-748106aa436b None None] MFA Rules not processed for user `29abe018b1dc4da69a55c75be6677fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:51.441398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b6d0f08e-e782-4414-a15a-748106aa436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:51.519535 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 947/1914] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:51.524123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a3778-2182-41ae-8fed-ac1274b1be31 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:51.592776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a3778-2182-41ae-8fed-ac1274b1be31 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:51.598476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308912d8-737a-4397-b43d-cc37da9002b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:51.599282 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308912d8-737a-4397-b43d-cc37da9002b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:51.600188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308912d8-737a-4397-b43d-cc37da9002b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:51.600561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308912d8-737a-4397-b43d-cc37da9002b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:51.601350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308912d8-737a-4397-b43d-cc37da9002b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:51.603434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308912d8-737a-4397-b43d-cc37da9002b6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:51.703397 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 968/1915] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:51 2022] POST /identity/v3/projects => generated 369 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:51.704732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0f08e-e782-4414-a15a-748106aa436b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:51.728577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0f08e-e782-4414-a15a-748106aa436b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:51.829578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785229f1-77c6-4725-88ae-d574f1db5934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:51.830257 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785229f1-77c6-4725-88ae-d574f1db5934 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:51.830619 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785229f1-77c6-4725-88ae-d574f1db5934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:51.830968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785229f1-77c6-4725-88ae-d574f1db5934 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:51.831858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785229f1-77c6-4725-88ae-d574f1db5934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:51.834667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785229f1-77c6-4725-88ae-d574f1db5934 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:51.955846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 948/1916] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:51 2022] POST /identity/v3/users => generated 291 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:51.957659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308912d8-737a-4397-b43d-cc37da9002b6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:51.964110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:51.970796 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308912d8-737a-4397-b43d-cc37da9002b6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:51.976286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c33bd9-4b1a-4fa5-9ed9-62082b7d5e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:51.977031 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c33bd9-4b1a-4fa5-9ed9-62082b7d5e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:51.977372 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c33bd9-4b1a-4fa5-9ed9-62082b7d5e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:51.982418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c33bd9-4b1a-4fa5-9ed9-62082b7d5e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:51.982418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c33bd9-4b1a-4fa5-9ed9-62082b7d5e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:51.982418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c33bd9-4b1a-4fa5-9ed9-62082b7d5e5d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:51.990877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 969/1917] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:51.995870 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785229f1-77c6-4725-88ae-d574f1db5934 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:52.006810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785229f1-77c6-4725-88ae-d574f1db5934 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:52.011810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96df96-4207-4c3e-8e60-18a5fc1d0bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:52.013010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96df96-4207-4c3e-8e60-18a5fc1d0bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:52.013217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96df96-4207-4c3e-8e60-18a5fc1d0bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:52.016957 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96df96-4207-4c3e-8e60-18a5fc1d0bb2 admin admin] PATH_INFO: `/v3/projects/166f10c2ada64c579d8dc17d24517323/users/d70f85e4d950405bb8ffb8bf517e4d21/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:52.031213 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96df96-4207-4c3e-8e60-18a5fc1d0bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=166f10c2ada64c579d8dc17d24517323, user_id=d70f85e4d950405bb8ffb8bf517e4d21, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:52.034170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96df96-4207-4c3e-8e60-18a5fc1d0bb2 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:52.160449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:52.167980 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 949/1918] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:51 2022] PUT /identity/v3/projects/166f10c2ada64c579d8dc17d24517323/users/d70f85e4d950405bb8ffb8bf517e4d21/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:52.171612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c33bd9-4b1a-4fa5-9ed9-62082b7d5e5d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:52.186711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c33bd9-4b1a-4fa5-9ed9-62082b7d5e5d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:52.279029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afbe40-b4f2-4e8b-b7eb-5f96f67131ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:52.280344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afbe40-b4f2-4e8b-b7eb-5f96f67131ab admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:52.280701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afbe40-b4f2-4e8b-b7eb-5f96f67131ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:52.281655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afbe40-b4f2-4e8b-b7eb-5f96f67131ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:52.282733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afbe40-b4f2-4e8b-b7eb-5f96f67131ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:52.285099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afbe40-b4f2-4e8b-b7eb-5f96f67131ab admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:52.300368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 970/1919] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:52.305104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96df96-4207-4c3e-8e60-18a5fc1d0bb2 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:52.316095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96df96-4207-4c3e-8e60-18a5fc1d0bb2 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:52.321222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f686208-a793-44d2-a335-338bced8dc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:52.321889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f686208-a793-44d2-a335-338bced8dc89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:52.324645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f686208-a793-44d2-a335-338bced8dc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:52.324645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f686208-a793-44d2-a335-338bced8dc89 admin admin] PATH_INFO: `/v3/projects/166f10c2ada64c579d8dc17d24517323/users/d70f85e4d950405bb8ffb8bf517e4d21/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:52.324645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f686208-a793-44d2-a335-338bced8dc89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=166f10c2ada64c579d8dc17d24517323, user_id=d70f85e4d950405bb8ffb8bf517e4d21, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:52.327152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f686208-a793-44d2-a335-338bced8dc89 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:52.337357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 950/1920] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:52 2022] PUT /identity/v3/projects/166f10c2ada64c579d8dc17d24517323/users/d70f85e4d950405bb8ffb8bf517e4d21/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:52.348199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf04d3-3419-4a55-bc87-3339bd81dfbc None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:52.348493 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf04d3-3419-4a55-bc87-3339bd81dfbc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:52.348778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf04d3-3419-4a55-bc87-3339bd81dfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:52.379378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f686208-a793-44d2-a335-338bced8dc89 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:52.381241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7f686208-a793-44d2-a335-338bced8dc89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:52.384785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-12cf04d3-3419-4a55-bc87-3339bd81dfbc None None] MFA Rules not processed for user `d70f85e4d950405bb8ffb8bf517e4d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:52.393712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-12cf04d3-3419-4a55-bc87-3339bd81dfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:52.483600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f686208-a793-44d2-a335-338bced8dc89 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:52.488892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:52.489583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:52.489867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:52.490187 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] PATH_INFO: `/v3/projects/6de15323f5af42d493b9f6f9dcd69b52` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:52.491391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6de15323f5af42d493b9f6f9dcd69b52)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:52.492663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 971/1921] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:52 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:52.493502 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:52.509193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf04d3-3419-4a55-bc87-3339bd81dfbc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:52.510710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-12cf04d3-3419-4a55-bc87-3339bd81dfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:52.585916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf04d3-3419-4a55-bc87-3339bd81dfbc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:52.590001 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d38b1-53ad-4ba7-8b85-8ede1e9194da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:52.591286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d38b1-53ad-4ba7-8b85-8ede1e9194da service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:52.591913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d38b1-53ad-4ba7-8b85-8ede1e9194da service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:52.591913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d38b1-53ad-4ba7-8b85-8ede1e9194da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:52.602478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d38b1-53ad-4ba7-8b85-8ede1e9194da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:52.604892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d38b1-53ad-4ba7-8b85-8ede1e9194da service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:52.609252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6de15323f5af42d493b9f6f9dcd69b52', 'request_id': 'req-a54f83e5-c345-47fe-b639-4f3b9cd2c096'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:52.615568 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6de15323f5af42d493b9f6f9dcd69b52', 'request_id': 'req-a54f83e5-c345-47fe-b639-4f3b9cd2c096'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:52.621241 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 972/1922] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:52 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:13:52.683201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6de15323f5af42d493b9f6f9dcd69b52', 'request_id': 'req-a54f83e5-c345-47fe-b639-4f3b9cd2c096'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:52.699042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] The token cache is being invalidate because project 6de15323f5af42d493b9f6f9dcd69b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:52.699404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:52.703285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a54f83e5-c345-47fe-b639-4f3b9cd2c096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:52.706066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 951/1923] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:52 2022] DELETE /identity/v3/projects/6de15323f5af42d493b9f6f9dcd69b52 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:52.708081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048791c-d35b-4ac4-b888-244a47a03802 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:52.708572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048791c-d35b-4ac4-b888-244a47a03802 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:52.709574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048791c-d35b-4ac4-b888-244a47a03802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:52.742880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5048791c-d35b-4ac4-b888-244a47a03802 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:52.744108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5048791c-d35b-4ac4-b888-244a47a03802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:52.761183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:52.830722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 973/1924] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:13:52.843824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af7abb-6975-42aa-ac65-2b7dd6527f6e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:52.844320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af7abb-6975-42aa-ac65-2b7dd6527f6e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:52.846014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af7abb-6975-42aa-ac65-2b7dd6527f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:52.917771 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-66af7abb-6975-42aa-ac65-2b7dd6527f6e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:52.919487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-66af7abb-6975-42aa-ac65-2b7dd6527f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:52.943603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 952/1925] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:52.948202 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048791c-d35b-4ac4-b888-244a47a03802 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:52.962213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048791c-d35b-4ac4-b888-244a47a03802 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:52.985957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf9b1c-e096-40dd-a4bc-e282ada563cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:52.986859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf9b1c-e096-40dd-a4bc-e282ada563cf admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:52.987349 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf9b1c-e096-40dd-a4bc-e282ada563cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:52.988024 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf9b1c-e096-40dd-a4bc-e282ada563cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:52.989037 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf9b1c-e096-40dd-a4bc-e282ada563cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:13:52.998498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf9b1c-e096-40dd-a4bc-e282ada563cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:52.998498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf9b1c-e096-40dd-a4bc-e282ada563cf admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:53.026374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af7abb-6975-42aa-ac65-2b7dd6527f6e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:53.031817 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 974/1926] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:13:53.042692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af7abb-6975-42aa-ac65-2b7dd6527f6e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:53.056089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465e7cb-d30f-4409-a1c9-9922d34153f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:53.071951 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465e7cb-d30f-4409-a1c9-9922d34153f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:53.072324 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465e7cb-d30f-4409-a1c9-9922d34153f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:53.072643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465e7cb-d30f-4409-a1c9-9922d34153f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:53.073558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465e7cb-d30f-4409-a1c9-9922d34153f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:53.092944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465e7cb-d30f-4409-a1c9-9922d34153f5 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:53.198458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 953/1927] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:53 2022] POST /identity/v3/projects => generated 371 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:53.199171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf9b1c-e096-40dd-a4bc-e282ada563cf admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:53.212174 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf9b1c-e096-40dd-a4bc-e282ada563cf admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:53.318631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af6ce3-3780-4ea2-b343-9e7b2950705c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:53.320625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6ce3-3780-4ea2-b343-9e7b2950705c admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:53.320625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6ce3-3780-4ea2-b343-9e7b2950705c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:53.320625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6ce3-3780-4ea2-b343-9e7b2950705c admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:53.320625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af6ce3-3780-4ea2-b343-9e7b2950705c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:53.322467 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af6ce3-3780-4ea2-b343-9e7b2950705c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:53.454381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 975/1928] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:53 2022] POST /identity/v3/users => generated 286 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:13:53.458867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465e7cb-d30f-4409-a1c9-9922d34153f5 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:53.471646 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465e7cb-d30f-4409-a1c9-9922d34153f5 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:53.476843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770bb12-bf99-40fd-a9e7-dc6f2098c10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:53.488230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770bb12-bf99-40fd-a9e7-dc6f2098c10d admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:53.488230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770bb12-bf99-40fd-a9e7-dc6f2098c10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:53.488230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770bb12-bf99-40fd-a9e7-dc6f2098c10d admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:53.488230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770bb12-bf99-40fd-a9e7-dc6f2098c10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:53.488230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770bb12-bf99-40fd-a9e7-dc6f2098c10d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:53.495117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 954/1929] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:53.496320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af6ce3-3780-4ea2-b343-9e7b2950705c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:53.508210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af6ce3-3780-4ea2-b343-9e7b2950705c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:53.514395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7859779-b0fd-471e-bfaa-c070db68ebe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:53.515170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859779-b0fd-471e-bfaa-c070db68ebe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:53.523159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859779-b0fd-471e-bfaa-c070db68ebe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:53.523159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859779-b0fd-471e-bfaa-c070db68ebe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:53.523159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7859779-b0fd-471e-bfaa-c070db68ebe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:53.523159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7859779-b0fd-471e-bfaa-c070db68ebe7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:53.537020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 976/1930] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:53.541017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770bb12-bf99-40fd-a9e7-dc6f2098c10d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:53.561169 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770bb12-bf99-40fd-a9e7-dc6f2098c10d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:53.567305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85693c13-00f1-49e8-b11b-5ca08d66ed5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:53.568753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85693c13-00f1-49e8-b11b-5ca08d66ed5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:53.569252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85693c13-00f1-49e8-b11b-5ca08d66ed5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:53.569888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85693c13-00f1-49e8-b11b-5ca08d66ed5d admin admin] PATH_INFO: `/v3/projects/85d8da2c21474c958de9039caa0b547d/users/a794c5d63c0f499db264113d64ae63b5/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:53.628749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85693c13-00f1-49e8-b11b-5ca08d66ed5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d8da2c21474c958de9039caa0b547d, user_id=a794c5d63c0f499db264113d64ae63b5, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:53.644433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85693c13-00f1-49e8-b11b-5ca08d66ed5d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:53.731982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 955/1931] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:53 2022] PUT /identity/v3/projects/85d8da2c21474c958de9039caa0b547d/users/a794c5d63c0f499db264113d64ae63b5/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:53.737207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1083006-c61b-43b2-87e7-89188384cb15 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:53.737678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1083006-c61b-43b2-87e7-89188384cb15 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:53.741173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1083006-c61b-43b2-87e7-89188384cb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:53.763882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e1083006-c61b-43b2-87e7-89188384cb15 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:53.765566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e1083006-c61b-43b2-87e7-89188384cb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:53.846749 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 977/1932] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:53.853947 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85693c13-00f1-49e8-b11b-5ca08d66ed5d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:53.855842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-85693c13-00f1-49e8-b11b-5ca08d66ed5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:53.871377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:53.930611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85693c13-00f1-49e8-b11b-5ca08d66ed5d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:53.938225 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b62a8d-89e3-42a9-9103-6368a306820d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:53.938225 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b62a8d-89e3-42a9-9103-6368a306820d service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:53.938225 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b62a8d-89e3-42a9-9103-6368a306820d service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:53.938225 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b62a8d-89e3-42a9-9103-6368a306820d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:53.952200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b62a8d-89e3-42a9-9103-6368a306820d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:53.954058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b62a8d-89e3-42a9-9103-6368a306820d service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:53.966793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 956/1933] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:13:55.656108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e3f3a-46e2-4a27-ace6-af7ee3563640 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:55.657075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e3f3a-46e2-4a27-ace6-af7ee3563640 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:55.657559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e3f3a-46e2-4a27-ace6-af7ee3563640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:55.742965 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-821e3f3a-46e2-4a27-ace6-af7ee3563640 None None] MFA Rules not processed for user `a794c5d63c0f499db264113d64ae63b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:55.745080 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-821e3f3a-46e2-4a27-ace6-af7ee3563640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:55.762211 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:55.831470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 978/1934] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:55.838720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b62a8d-89e3-42a9-9103-6368a306820d service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:55.840382 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-23b62a8d-89e3-42a9-9103-6368a306820d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:55.901798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b62a8d-89e3-42a9-9103-6368a306820d service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:55.905594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbd4be-e4f2-4b39-98b2-28f22a76670f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:55.906212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbd4be-e4f2-4b39-98b2-28f22a76670f service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:55.906522 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbd4be-e4f2-4b39-98b2-28f22a76670f service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:55.906793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbd4be-e4f2-4b39-98b2-28f22a76670f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:55.915732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbd4be-e4f2-4b39-98b2-28f22a76670f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:55.917238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbd4be-e4f2-4b39-98b2-28f22a76670f service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:55.926290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 957/1935] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:56.521330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e3f3a-46e2-4a27-ace6-af7ee3563640 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:56.522967 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-821e3f3a-46e2-4a27-ace6-af7ee3563640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:56.538239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e3f3a-46e2-4a27-ace6-af7ee3563640 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:56.543180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:56.543994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:56.549424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:56.549424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] PATH_INFO: `/v3/users/29abe018b1dc4da69a55c75be6677fcb` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:56.549424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29abe018b1dc4da69a55c75be6677fcb)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:56.549424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:56.731097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29abe018b1dc4da69a55c75be6677fcb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:56.732174 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29abe018b1dc4da69a55c75be6677fcb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:56.739655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29abe018b1dc4da69a55c75be6677fcb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:56.805442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29abe018b1dc4da69a55c75be6677fcb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:56.806016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29abe018b1dc4da69a55c75be6677fcb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:56.824739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 979/1936] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:56 2022] DELETE /identity/v3/users/29abe018b1dc4da69a55c75be6677fcb => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:56.829940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047f56a-8f6b-4484-ac2a-0751250f4ce7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:56.830302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047f56a-8f6b-4484-ac2a-0751250f4ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:56.830651 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047f56a-8f6b-4484-ac2a-0751250f4ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:56.852709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0047f56a-8f6b-4484-ac2a-0751250f4ce7 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:56.854058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0047f56a-8f6b-4484-ac2a-0751250f4ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:56.916525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 958/1937] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:56.926603 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:56.931555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:56.991311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaff08-8f85-4d46-bec3-02f0ec279358 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:56.995145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f2e68-5d4e-4736-800c-695de933fc8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:56.995967 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2e68-5d4e-4736-800c-695de933fc8e service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:56.996251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2e68-5d4e-4736-800c-695de933fc8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:56.996531 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2e68-5d4e-4736-800c-695de933fc8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:57.006454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f2e68-5d4e-4736-800c-695de933fc8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:57.011849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f2e68-5d4e-4736-800c-695de933fc8e service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:57.019266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 980/1938] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:13:57.435929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047f56a-8f6b-4484-ac2a-0751250f4ce7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:57.437667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0047f56a-8f6b-4484-ac2a-0751250f4ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:57.452834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047f56a-8f6b-4484-ac2a-0751250f4ce7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:57.459914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:57.461139 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:57.461669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:57.462173 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] PATH_INFO: `/v3/projects/e2b32fcdd0ad41ea9a821bc81a0f1d03` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:57.463680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b32fcdd0ad41ea9a821bc81a0f1d03)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:57.465935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:57.591814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b32fcdd0ad41ea9a821bc81a0f1d03', 'request_id': 'req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.594369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:57.601144 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b32fcdd0ad41ea9a821bc81a0f1d03', 'request_id': 'req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.663981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b32fcdd0ad41ea9a821bc81a0f1d03', 'request_id': 'req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.678921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] The token cache is being invalidate because project e2b32fcdd0ad41ea9a821bc81a0f1d03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:57.679222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.680104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8fcedd62-de58-4167-9d0e-4dd2c2c308c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:57.686574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 959/1939] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:57 2022] DELETE /identity/v3/projects/e2b32fcdd0ad41ea9a821bc81a0f1d03 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:57.689702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f2e68-5d4e-4736-800c-695de933fc8e service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:57.703385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f2e68-5d4e-4736-800c-695de933fc8e service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:57.736993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:57.773400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:57.774170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:57.774523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:57.775040 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] PATH_INFO: `/v3/users/d70f85e4d950405bb8ffb8bf517e4d21` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:57.776361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d70f85e4d950405bb8ffb8bf517e4d21)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:57.778577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:58.023331 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd70f85e4d950405bb8ffb8bf517e4d21'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:58.024062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd70f85e4d950405bb8ffb8bf517e4d21'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:58.032435 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd70f85e4d950405bb8ffb8bf517e4d21'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:58.119219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd70f85e4d950405bb8ffb8bf517e4d21'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:58.120595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd70f85e4d950405bb8ffb8bf517e4d21'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:58.143568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 981/1940] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:13:57 2022] DELETE /identity/v3/users/d70f85e4d950405bb8ffb8bf517e4d21 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:13:58.803897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a4618-0517-4d53-a572-38a404baca04 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:58.804439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a4618-0517-4d53-a572-38a404baca04 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:58.804439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a4618-0517-4d53-a572-38a404baca04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:58.824230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-fa4a4618-0517-4d53-a572-38a404baca04 None None] MFA Rules not processed for user `a794c5d63c0f499db264113d64ae63b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:58.825508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fa4a4618-0517-4d53-a572-38a404baca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:58.839668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:58.842213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:58.969269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 960/1941] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:13:58.982284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3881f-5abc-4bf6-9784-140ffd2a52ec admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:58.992334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:58.995821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:58.995821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:58.995821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] PATH_INFO: `/v3/projects/166f10c2ada64c579d8dc17d24517323` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:58.996803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] RBAC: Authorizing `identity:delete_project(project_id=166f10c2ada64c579d8dc17d24517323)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:58.998199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.142911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:59.143955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '166f10c2ada64c579d8dc17d24517323', 'request_id': 'req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:59.150147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '166f10c2ada64c579d8dc17d24517323', 'request_id': 'req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:59.156528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '166f10c2ada64c579d8dc17d24517323', 'request_id': 'req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:59.296263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] The token cache is being invalidate because project 166f10c2ada64c579d8dc17d24517323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:13:59.296557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:59.297268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-193ad814-75d3-47ba-b9dd-ce76a1f1a27f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:13:59.300922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 982/1942] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:13:58 2022] DELETE /identity/v3/projects/166f10c2ada64c579d8dc17d24517323 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:13:59.304240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208f6b6-fd39-45e3-af71-6f6c15453eac None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.304534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208f6b6-fd39-45e3-af71-6f6c15453eac None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.304816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208f6b6-fd39-45e3-af71-6f6c15453eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.334966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7208f6b6-fd39-45e3-af71-6f6c15453eac None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:59.336554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7208f6b6-fd39-45e3-af71-6f6c15453eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:59.406610 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 961/1943] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:59.416612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194882-f405-4191-b463-521895e775c2 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.416922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194882-f405-4191-b463-521895e775c2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.417606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194882-f405-4191-b463-521895e775c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.444655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-38194882-f405-4191-b463-521895e775c2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:13:59.444655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-38194882-f405-4191-b463-521895e775c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:13:59.457451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 983/1944] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:13:59.458729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208f6b6-fd39-45e3-af71-6f6c15453eac None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.471000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208f6b6-fd39-45e3-af71-6f6c15453eac None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.476014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c67c3-7ea2-40a2-999e-69f124549fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.476729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c67c3-7ea2-40a2-999e-69f124549fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.477046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c67c3-7ea2-40a2-999e-69f124549fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.477329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c67c3-7ea2-40a2-999e-69f124549fd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.478061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c67c3-7ea2-40a2-999e-69f124549fd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:13:59.478370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c67c3-7ea2-40a2-999e-69f124549fd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.479998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c67c3-7ea2-40a2-999e-69f124549fd9 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.493390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 962/1945] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:13:59.498194 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38194882-f405-4191-b463-521895e775c2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.510337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38194882-f405-4191-b463-521895e775c2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.515077 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73698d-6efc-4521-993f-a0a0415d2713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.515706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73698d-6efc-4521-993f-a0a0415d2713 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.517520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73698d-6efc-4521-993f-a0a0415d2713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.517520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73698d-6efc-4521-993f-a0a0415d2713 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.517520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73698d-6efc-4521-993f-a0a0415d2713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.519496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73698d-6efc-4521-993f-a0a0415d2713 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.603499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 984/1946] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:13:59 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:59.605537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c67c3-7ea2-40a2-999e-69f124549fd9 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.639684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c67c3-7ea2-40a2-999e-69f124549fd9 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.711017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d86ab8-8feb-44c5-a13c-ab89bacb957a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.711655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d86ab8-8feb-44c5-a13c-ab89bacb957a admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.712028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d86ab8-8feb-44c5-a13c-ab89bacb957a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.712491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d86ab8-8feb-44c5-a13c-ab89bacb957a admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.713224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d86ab8-8feb-44c5-a13c-ab89bacb957a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.715281 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d86ab8-8feb-44c5-a13c-ab89bacb957a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.806235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 963/1947] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:13:59 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:13:59.809964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73698d-6efc-4521-993f-a0a0415d2713 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.820273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73698d-6efc-4521-993f-a0a0415d2713 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.824955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb34af4-fa45-41e7-af0c-480157f6ffcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.825502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb34af4-fa45-41e7-af0c-480157f6ffcf admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.825787 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb34af4-fa45-41e7-af0c-480157f6ffcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.826066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb34af4-fa45-41e7-af0c-480157f6ffcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.826767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb34af4-fa45-41e7-af0c-480157f6ffcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.828361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb34af4-fa45-41e7-af0c-480157f6ffcf admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.838300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 985/1948] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:59.842323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d86ab8-8feb-44c5-a13c-ab89bacb957a admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.847793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:13:59.855235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d86ab8-8feb-44c5-a13c-ab89bacb957a admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.859873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0a8c4-c2a0-4c96-805d-949857754026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.860598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0a8c4-c2a0-4c96-805d-949857754026 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.860799 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0a8c4-c2a0-4c96-805d-949857754026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.861112 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0a8c4-c2a0-4c96-805d-949857754026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.861816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0a8c4-c2a0-4c96-805d-949857754026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.863408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0a8c4-c2a0-4c96-805d-949857754026 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.874722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 964/1949] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:13:59.878981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb34af4-fa45-41e7-af0c-480157f6ffcf admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:13:59.889458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb34af4-fa45-41e7-af0c-480157f6ffcf admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:13:59.894082 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc3eed-cba3-4ce3-802a-c7162688ca29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:13:59.894741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc3eed-cba3-4ce3-802a-c7162688ca29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:13:59.895024 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc3eed-cba3-4ce3-802a-c7162688ca29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:13:59.895295 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc3eed-cba3-4ce3-802a-c7162688ca29 admin admin] PATH_INFO: `/v3/projects/b51d9aef87bf45e8bf8063a1b49dc368/users/cf5d7559b30940979402cee6294c612f/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:13:59.910067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc3eed-cba3-4ce3-802a-c7162688ca29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b51d9aef87bf45e8bf8063a1b49dc368, user_id=cf5d7559b30940979402cee6294c612f, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:13:59.913300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc3eed-cba3-4ce3-802a-c7162688ca29 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:13:59.999994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 986/1950] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:13:59 2022] PUT /identity/v3/projects/b51d9aef87bf45e8bf8063a1b49dc368/users/cf5d7559b30940979402cee6294c612f/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:00.011404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4e3e-f64a-43dd-8735-d132b51cfc56 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.011697 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4e3e-f64a-43dd-8735-d132b51cfc56 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.011978 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4e3e-f64a-43dd-8735-d132b51cfc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.042039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-ceed4e3e-f64a-43dd-8735-d132b51cfc56 None None] MFA Rules not processed for user `cf5d7559b30940979402cee6294c612f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:00.043397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ceed4e3e-f64a-43dd-8735-d132b51cfc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:00.108137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 965/1951] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:00.112877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc3eed-cba3-4ce3-802a-c7162688ca29 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.179010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc3eed-cba3-4ce3-802a-c7162688ca29 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.183685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176bbe3-2551-4302-a082-2c6e7ea14110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.184310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176bbe3-2551-4302-a082-2c6e7ea14110 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.184595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176bbe3-2551-4302-a082-2c6e7ea14110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.184872 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176bbe3-2551-4302-a082-2c6e7ea14110 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.185584 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176bbe3-2551-4302-a082-2c6e7ea14110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.187656 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176bbe3-2551-4302-a082-2c6e7ea14110 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.280624 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 987/1952] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:00 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:00.284769 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4e3e-f64a-43dd-8735-d132b51cfc56 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.296730 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4e3e-f64a-43dd-8735-d132b51cfc56 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.355621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9f65e-5bc0-4c58-8c6f-5ae17268a73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.356955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9f65e-5bc0-4c58-8c6f-5ae17268a73c admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.357276 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9f65e-5bc0-4c58-8c6f-5ae17268a73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.358270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9f65e-5bc0-4c58-8c6f-5ae17268a73c admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.358922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9f65e-5bc0-4c58-8c6f-5ae17268a73c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.360860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9f65e-5bc0-4c58-8c6f-5ae17268a73c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.433956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 966/1953] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:00 2022] POST /identity/v3/users => generated 295 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:00.438083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176bbe3-2551-4302-a082-2c6e7ea14110 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.448786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176bbe3-2551-4302-a082-2c6e7ea14110 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.453660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799080a3-af4d-4f12-93b3-b2943a117c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.454290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799080a3-af4d-4f12-93b3-b2943a117c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.454579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799080a3-af4d-4f12-93b3-b2943a117c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.454859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799080a3-af4d-4f12-93b3-b2943a117c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.455573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799080a3-af4d-4f12-93b3-b2943a117c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.458023 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799080a3-af4d-4f12-93b3-b2943a117c43 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.471596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 988/1954] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:00.473886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9f65e-5bc0-4c58-8c6f-5ae17268a73c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.484078 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9f65e-5bc0-4c58-8c6f-5ae17268a73c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.488778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13ce09-9897-42df-83f7-1740c2398b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.489482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13ce09-9897-42df-83f7-1740c2398b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.489773 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13ce09-9897-42df-83f7-1740c2398b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.490058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13ce09-9897-42df-83f7-1740c2398b5b admin admin] PATH_INFO: `/v3/projects/8a32677ba87348e1be912309d02d6613/users/9f9cd42445bf440fb1312c7df302d6b5/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.506456 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13ce09-9897-42df-83f7-1740c2398b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a32677ba87348e1be912309d02d6613, user_id=9f9cd42445bf440fb1312c7df302d6b5, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.509894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13ce09-9897-42df-83f7-1740c2398b5b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.627538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 967/1955] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:00 2022] PUT /identity/v3/projects/8a32677ba87348e1be912309d02d6613/users/9f9cd42445bf440fb1312c7df302d6b5/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:00.632164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799080a3-af4d-4f12-93b3-b2943a117c43 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.655643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799080a3-af4d-4f12-93b3-b2943a117c43 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.719803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:00.733805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02325998-68af-457d-ab2a-e446a6d50ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.734496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02325998-68af-457d-ab2a-e446a6d50ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.734784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02325998-68af-457d-ab2a-e446a6d50ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.735063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02325998-68af-457d-ab2a-e446a6d50ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.739264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02325998-68af-457d-ab2a-e446a6d50ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.739264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02325998-68af-457d-ab2a-e446a6d50ad8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.748741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 989/1956] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:00.752924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13ce09-9897-42df-83f7-1740c2398b5b admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.778060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13ce09-9897-42df-83f7-1740c2398b5b admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.783783 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f3087-a3b3-4f3d-a6d6-a968dfe7ea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.784473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f3087-a3b3-4f3d-a6d6-a968dfe7ea31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.784832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f3087-a3b3-4f3d-a6d6-a968dfe7ea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.785152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f3087-a3b3-4f3d-a6d6-a968dfe7ea31 admin admin] PATH_INFO: `/v3/projects/8a32677ba87348e1be912309d02d6613/users/9f9cd42445bf440fb1312c7df302d6b5/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.788051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f3087-a3b3-4f3d-a6d6-a968dfe7ea31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a32677ba87348e1be912309d02d6613, user_id=9f9cd42445bf440fb1312c7df302d6b5, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:00.790821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f3087-a3b3-4f3d-a6d6-a968dfe7ea31 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:00.808763 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 968/1957] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:00 2022] PUT /identity/v3/projects/8a32677ba87348e1be912309d02d6613/users/9f9cd42445bf440fb1312c7df302d6b5/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:00.813061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfa133-b0e1-46bc-9dfd-1705a93919d3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.813357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfa133-b0e1-46bc-9dfd-1705a93919d3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.813719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfa133-b0e1-46bc-9dfd-1705a93919d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:00.844688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0fcfa133-b0e1-46bc-9dfd-1705a93919d3 None None] MFA Rules not processed for user `9f9cd42445bf440fb1312c7df302d6b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:00.845964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0fcfa133-b0e1-46bc-9dfd-1705a93919d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:00.921712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 990/1958] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:00.935807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f3087-a3b3-4f3d-a6d6-a968dfe7ea31 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:00.937525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-235f3087-a3b3-4f3d-a6d6-a968dfe7ea31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:00.973964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:00.995150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f3087-a3b3-4f3d-a6d6-a968dfe7ea31 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:00.998761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a3479-5e6d-4ab2-a893-69c45d42093f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:00.999375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a3479-5e6d-4ab2-a893-69c45d42093f service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:00.999656 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a3479-5e6d-4ab2-a893-69c45d42093f service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:00.999934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a3479-5e6d-4ab2-a893-69c45d42093f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:01.010861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a3479-5e6d-4ab2-a893-69c45d42093f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:01.012301 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a3479-5e6d-4ab2-a893-69c45d42093f service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:01.022061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 969/1959] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:02.045976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfa133-b0e1-46bc-9dfd-1705a93919d3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:02.056158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfa133-b0e1-46bc-9dfd-1705a93919d3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:02.060925 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75e385-ab4f-4838-a348-4209b909bc0d tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f9cd42445bf440fb1312c7df302d6b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a32677ba87348e1be912309d02d6613', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:02.061574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75e385-ab4f-4838-a348-4209b909bc0d tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:02.061916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75e385-ab4f-4838-a348-4209b909bc0d tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:02.062125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75e385-ab4f-4838-a348-4209b909bc0d tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:02.062827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75e385-ab4f-4838-a348-4209b909bc0d tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:02.063140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75e385-ab4f-4838-a348-4209b909bc0d tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:02.064746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75e385-ab4f-4838-a348-4209b909bc0d tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:02.076664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 991/1960] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:02.080518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a3479-5e6d-4ab2-a893-69c45d42093f service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:02.089606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a3479-5e6d-4ab2-a893-69c45d42093f service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:02.094556 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7aa1b-a52e-4be7-b3e9-4355be94b1f6 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f9cd42445bf440fb1312c7df302d6b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a32677ba87348e1be912309d02d6613', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:02.095160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7aa1b-a52e-4be7-b3e9-4355be94b1f6 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:02.095440 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7aa1b-a52e-4be7-b3e9-4355be94b1f6 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:02.095718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7aa1b-a52e-4be7-b3e9-4355be94b1f6 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:02.096409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7aa1b-a52e-4be7-b3e9-4355be94b1f6 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:02.098311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7aa1b-a52e-4be7-b3e9-4355be94b1f6 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:02.199141 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 970/1961] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:02 2022] POST /identity/v3/projects => generated 353 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:02.203305 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75e385-ab4f-4838-a348-4209b909bc0d tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:02.213909 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75e385-ab4f-4838-a348-4209b909bc0d tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:02.267758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e0894-c978-4be5-9ab4-0304afba4e74 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f9cd42445bf440fb1312c7df302d6b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a32677ba87348e1be912309d02d6613', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:02.268390 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e0894-c978-4be5-9ab4-0304afba4e74 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:02.268650 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e0894-c978-4be5-9ab4-0304afba4e74 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:02.268922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e0894-c978-4be5-9ab4-0304afba4e74 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:02.269639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e0894-c978-4be5-9ab4-0304afba4e74 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:02.269943 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e0894-c978-4be5-9ab4-0304afba4e74 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:02.272068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e0894-c978-4be5-9ab4-0304afba4e74 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:02.283942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 992/1962] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:02.324572 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7aa1b-a52e-4be7-b3e9-4355be94b1f6 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:02.335383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7aa1b-a52e-4be7-b3e9-4355be94b1f6 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:02.340010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f9cd42445bf440fb1312c7df302d6b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a32677ba87348e1be912309d02d6613', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:02.340646 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:02.340933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:02.341253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] PATH_INFO: `/v3/projects/3ad97f8aa1854754ba0e6dc93beb3838` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:02.342686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3ad97f8aa1854754ba0e6dc93beb3838)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:02.344590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:02.449439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ad97f8aa1854754ba0e6dc93beb3838', 'request_id': 'req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.454840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ad97f8aa1854754ba0e6dc93beb3838', 'request_id': 'req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.530472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ad97f8aa1854754ba0e6dc93beb3838', 'request_id': 'req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.542953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] The token cache is being invalidate because project 3ad97f8aa1854754ba0e6dc93beb3838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:02.543240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.543877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af14fcb6-3c8c-445e-910b-c4f2bb728f1b tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:02.545448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 971/1963] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:02 2022] DELETE /identity/v3/projects/3ad97f8aa1854754ba0e6dc93beb3838 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:03.193982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e0894-c978-4be5-9ab4-0304afba4e74 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:03.196347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8b4e0894-c978-4be5-9ab4-0304afba4e74 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:03.245684 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:03.261531 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e0894-c978-4be5-9ab4-0304afba4e74 tempest-VolumeQuotaClassesTest-33380258 tempest-VolumeQuotaClassesTest-33380258-project-admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:03.317178 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:03.317821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:03.318107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:03.318388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] PATH_INFO: `/v3/users/cf5d7559b30940979402cee6294c612f` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:03.320049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf5d7559b30940979402cee6294c612f)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:03.322004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:03.480296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf5d7559b30940979402cee6294c612f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:03.480948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf5d7559b30940979402cee6294c612f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:03.487931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf5d7559b30940979402cee6294c612f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:03.558573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf5d7559b30940979402cee6294c612f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:03.559126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf5d7559b30940979402cee6294c612f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:03.577293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 993/1964] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:03 2022] DELETE /identity/v3/users/cf5d7559b30940979402cee6294c612f => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:03.581846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34baf40-326a-4e9d-99ce-6572e76ab6b9 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:03.582326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34baf40-326a-4e9d-99ce-6572e76ab6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:03.582421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34baf40-326a-4e9d-99ce-6572e76ab6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:03.601887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a34baf40-326a-4e9d-99ce-6572e76ab6b9 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:03.603101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a34baf40-326a-4e9d-99ce-6572e76ab6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:03.663153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 972/1965] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:03.674014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:03.675443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:03.736033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929a966-10a5-4d50-a9e5-f75afea43a17 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:03.739714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4154c-1a19-450a-9871-d60974213c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:03.740412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4154c-1a19-450a-9871-d60974213c8b service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:03.746204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4154c-1a19-450a-9871-d60974213c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:03.746204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4154c-1a19-450a-9871-d60974213c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:03.754984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4154c-1a19-450a-9871-d60974213c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:03.756413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4154c-1a19-450a-9871-d60974213c8b service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:03.766738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 994/1966] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:04.177717 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34baf40-326a-4e9d-99ce-6572e76ab6b9 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:04.179209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a34baf40-326a-4e9d-99ce-6572e76ab6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:04.192051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34baf40-326a-4e9d-99ce-6572e76ab6b9 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:04.209835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:04.211080 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:04.211370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:04.211648 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] PATH_INFO: `/v3/projects/b51d9aef87bf45e8bf8063a1b49dc368` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:04.212865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b51d9aef87bf45e8bf8063a1b49dc368)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:04.215404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:04.301981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:04.313887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b51d9aef87bf45e8bf8063a1b49dc368', 'request_id': 'req-47d18334-8937-491c-8bfa-c6df18c63dc7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:04.319819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b51d9aef87bf45e8bf8063a1b49dc368', 'request_id': 'req-47d18334-8937-491c-8bfa-c6df18c63dc7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:04.380923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b51d9aef87bf45e8bf8063a1b49dc368', 'request_id': 'req-47d18334-8937-491c-8bfa-c6df18c63dc7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:04.393096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] The token cache is being invalidate because project b51d9aef87bf45e8bf8063a1b49dc368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:04.393387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:04.394107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:04.395684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 973/1967] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:04 2022] DELETE /identity/v3/projects/b51d9aef87bf45e8bf8063a1b49dc368 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:04.399748 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4154c-1a19-450a-9871-d60974213c8b service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:04.409317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4154c-1a19-450a-9871-d60974213c8b service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:04.422284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:04.468114 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4e527-063d-4e0e-9ec7-2d60cc6bad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:04.468763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4e527-063d-4e0e-9ec7-2d60cc6bad23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:04.469130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4e527-063d-4e0e-9ec7-2d60cc6bad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:04.469412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4e527-063d-4e0e-9ec7-2d60cc6bad23 admin admin] PATH_INFO: `/v3/users/9f9cd42445bf440fb1312c7df302d6b5` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:04.470644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4e527-063d-4e0e-9ec7-2d60cc6bad23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f9cd42445bf440fb1312c7df302d6b5)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:04.472541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4e527-063d-4e0e-9ec7-2d60cc6bad23 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:04.652313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3fa4e527-063d-4e0e-9ec7-2d60cc6bad23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f9cd42445bf440fb1312c7df302d6b5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:04.652899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3fa4e527-063d-4e0e-9ec7-2d60cc6bad23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f9cd42445bf440fb1312c7df302d6b5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:04.659882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3fa4e527-063d-4e0e-9ec7-2d60cc6bad23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f9cd42445bf440fb1312c7df302d6b5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:04.734724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3fa4e527-063d-4e0e-9ec7-2d60cc6bad23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f9cd42445bf440fb1312c7df302d6b5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:04.735365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3fa4e527-063d-4e0e-9ec7-2d60cc6bad23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f9cd42445bf440fb1312c7df302d6b5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:04.754516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 995/1968] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:04 2022] DELETE /identity/v3/users/9f9cd42445bf440fb1312c7df302d6b5 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:05.167532 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:05.169148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:05.182331 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d18334-8937-491c-8bfa-c6df18c63dc7 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:05.246716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:05.247381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:05.247699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:05.247978 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] PATH_INFO: `/v3/projects/8a32677ba87348e1be912309d02d6613` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:05.249230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a32677ba87348e1be912309d02d6613)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:05.251266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:05.364133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a32677ba87348e1be912309d02d6613', 'request_id': 'req-2560712c-a7f5-491f-ab35-7b5066b454ad'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:05.370460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a32677ba87348e1be912309d02d6613', 'request_id': 'req-2560712c-a7f5-491f-ab35-7b5066b454ad'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:05.451301 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a32677ba87348e1be912309d02d6613', 'request_id': 'req-2560712c-a7f5-491f-ab35-7b5066b454ad'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:05.465914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] The token cache is being invalidate because project 8a32677ba87348e1be912309d02d6613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:05.466212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:05.467165 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2560712c-a7f5-491f-ab35-7b5066b454ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:05.468742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 974/1969] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:05 2022] DELETE /identity/v3/projects/8a32677ba87348e1be912309d02d6613 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:05.474162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96d886-8e03-4393-8d5e-757f5d094358 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:05.474648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96d886-8e03-4393-8d5e-757f5d094358 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:05.474849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96d886-8e03-4393-8d5e-757f5d094358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:05.497015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2c96d886-8e03-4393-8d5e-757f5d094358 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:05.498268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2c96d886-8e03-4393-8d5e-757f5d094358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:05.581582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 996/1970] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:05.583256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb791a-b0a5-42c6-9984-898483db987b None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:05.583547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb791a-b0a5-42c6-9984-898483db987b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:05.583966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb791a-b0a5-42c6-9984-898483db987b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:05.605570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1cbb791a-b0a5-42c6-9984-898483db987b None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:05.632188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1cbb791a-b0a5-42c6-9984-898483db987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:05.632188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 975/1971] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:05.640840 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96d886-8e03-4393-8d5e-757f5d094358 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:05.652856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96d886-8e03-4393-8d5e-757f5d094358 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:05.657628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a794884-d6f1-48fb-95e0-4a08967759e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:05.658251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a794884-d6f1-48fb-95e0-4a08967759e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:05.658552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a794884-d6f1-48fb-95e0-4a08967759e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:05.658810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a794884-d6f1-48fb-95e0-4a08967759e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:05.659570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a794884-d6f1-48fb-95e0-4a08967759e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:05.659874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a794884-d6f1-48fb-95e0-4a08967759e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:05.661545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a794884-d6f1-48fb-95e0-4a08967759e6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:05.675952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 997/1972] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:05.698700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb791a-b0a5-42c6-9984-898483db987b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:05.698700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb791a-b0a5-42c6-9984-898483db987b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:05.698700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c802c56-6e8b-46f7-ac3e-ef4ef7dbfce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:05.699162 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c802c56-6e8b-46f7-ac3e-ef4ef7dbfce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:05.699365 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c802c56-6e8b-46f7-ac3e-ef4ef7dbfce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:05.699710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c802c56-6e8b-46f7-ac3e-ef4ef7dbfce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:05.700519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c802c56-6e8b-46f7-ac3e-ef4ef7dbfce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:05.702672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c802c56-6e8b-46f7-ac3e-ef4ef7dbfce9 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:05.887879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 976/1973] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:05 2022] POST /identity/v3/projects => generated 377 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:05.892011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a794884-d6f1-48fb-95e0-4a08967759e6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:05.911017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a794884-d6f1-48fb-95e0-4a08967759e6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:05.948833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:05.991049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24b99a-cf21-4047-9ce7-92a844fa2813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:05.991681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24b99a-cf21-4047-9ce7-92a844fa2813 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:05.991965 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24b99a-cf21-4047-9ce7-92a844fa2813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:05.992745 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24b99a-cf21-4047-9ce7-92a844fa2813 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:05.993487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24b99a-cf21-4047-9ce7-92a844fa2813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:05.995443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24b99a-cf21-4047-9ce7-92a844fa2813 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:06.107831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 998/1974] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:05 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:06.112043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c802c56-6e8b-46f7-ac3e-ef4ef7dbfce9 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:06.122959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c802c56-6e8b-46f7-ac3e-ef4ef7dbfce9 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:06.127635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94513c2a-2b28-4cf5-8054-f6e750290df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:06.128244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94513c2a-2b28-4cf5-8054-f6e750290df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:06.128530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94513c2a-2b28-4cf5-8054-f6e750290df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:06.128805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94513c2a-2b28-4cf5-8054-f6e750290df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:06.129547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94513c2a-2b28-4cf5-8054-f6e750290df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:06.131133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94513c2a-2b28-4cf5-8054-f6e750290df8 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:06.142893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 977/1975] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:06.146686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24b99a-cf21-4047-9ce7-92a844fa2813 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:06.157057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24b99a-cf21-4047-9ce7-92a844fa2813 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:06.161679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d0ace-88b5-418e-a49d-a4a08ddbd903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:06.162312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d0ace-88b5-418e-a49d-a4a08ddbd903 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:06.162593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d0ace-88b5-418e-a49d-a4a08ddbd903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:06.162871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d0ace-88b5-418e-a49d-a4a08ddbd903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:06.163571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d0ace-88b5-418e-a49d-a4a08ddbd903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:06.165523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d0ace-88b5-418e-a49d-a4a08ddbd903 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:06.177101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 999/1976] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:06.181827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94513c2a-2b28-4cf5-8054-f6e750290df8 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:06.192553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94513c2a-2b28-4cf5-8054-f6e750290df8 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:06.201708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e0bae-add2-4242-9219-cfe074488621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:06.201708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e0bae-add2-4242-9219-cfe074488621 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:06.201708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e0bae-add2-4242-9219-cfe074488621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:06.201708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e0bae-add2-4242-9219-cfe074488621 admin admin] PATH_INFO: `/v3/projects/d39054ef8f67447d8f19ba23505d1b37/users/a59b8216aa214553a442c1f897fd7526/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:06.210005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:06.217508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e0bae-add2-4242-9219-cfe074488621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39054ef8f67447d8f19ba23505d1b37, user_id=a59b8216aa214553a442c1f897fd7526, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:06.220472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e0bae-add2-4242-9219-cfe074488621 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:06.342279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 978/1977] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:06 2022] PUT /identity/v3/projects/d39054ef8f67447d8f19ba23505d1b37/users/a59b8216aa214553a442c1f897fd7526/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:06.354733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9538ac9-1fed-403e-b53c-2251d4694244 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:06.355031 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9538ac9-1fed-403e-b53c-2251d4694244 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:06.355314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9538ac9-1fed-403e-b53c-2251d4694244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:06.383722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a9538ac9-1fed-403e-b53c-2251d4694244 None None] MFA Rules not processed for user `a59b8216aa214553a442c1f897fd7526`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:06.385017 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a9538ac9-1fed-403e-b53c-2251d4694244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:06.450240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1000/1978] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:06.455710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e0bae-add2-4242-9219-cfe074488621 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:06.517598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e0bae-add2-4242-9219-cfe074488621 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:06.522266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a7f3f-e5c8-49cd-9076-431c36b1bd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:06.522880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a7f3f-e5c8-49cd-9076-431c36b1bd67 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:06.523161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a7f3f-e5c8-49cd-9076-431c36b1bd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:06.523981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a7f3f-e5c8-49cd-9076-431c36b1bd67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:06.524658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a7f3f-e5c8-49cd-9076-431c36b1bd67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:06.526589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a7f3f-e5c8-49cd-9076-431c36b1bd67 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:06.598671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 979/1979] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:06 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:06.602855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9538ac9-1fed-403e-b53c-2251d4694244 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:06.613659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9538ac9-1fed-403e-b53c-2251d4694244 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:06.670511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0f79b-1e23-4229-bf20-5df15473752a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:06.671134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0f79b-1e23-4229-bf20-5df15473752a admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:06.671416 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0f79b-1e23-4229-bf20-5df15473752a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:06.671692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0f79b-1e23-4229-bf20-5df15473752a admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:06.672375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0f79b-1e23-4229-bf20-5df15473752a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:06.674332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0f79b-1e23-4229-bf20-5df15473752a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:06.791057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1001/1980] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:06 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:06.792342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a7f3f-e5c8-49cd-9076-431c36b1bd67 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:06.817835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a7f3f-e5c8-49cd-9076-431c36b1bd67 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:06.825948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ae8bd-2d87-47fd-87a3-38ed1ce46ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:06.825948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ae8bd-2d87-47fd-87a3-38ed1ce46ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:06.825948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ae8bd-2d87-47fd-87a3-38ed1ce46ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:06.825948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ae8bd-2d87-47fd-87a3-38ed1ce46ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:06.826403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ae8bd-2d87-47fd-87a3-38ed1ce46ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:06.827802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ae8bd-2d87-47fd-87a3-38ed1ce46ae4 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:06.842060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 980/1981] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:06.844627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0f79b-1e23-4229-bf20-5df15473752a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:06.855508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0f79b-1e23-4229-bf20-5df15473752a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:06.860852 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bffe95-030d-4f3e-b1be-1cbb9f58d948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:06.861578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bffe95-030d-4f3e-b1be-1cbb9f58d948 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:06.861868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bffe95-030d-4f3e-b1be-1cbb9f58d948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:06.862143 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bffe95-030d-4f3e-b1be-1cbb9f58d948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:06.862840 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bffe95-030d-4f3e-b1be-1cbb9f58d948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:06.864451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bffe95-030d-4f3e-b1be-1cbb9f58d948 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:06.875933 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1002/1982] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:06.879841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ae8bd-2d87-47fd-87a3-38ed1ce46ae4 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:06.890866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ae8bd-2d87-47fd-87a3-38ed1ce46ae4 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:06.896014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d691d00-16b3-44a8-a2f1-ac538157c313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:06.896677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d691d00-16b3-44a8-a2f1-ac538157c313 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:06.897108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d691d00-16b3-44a8-a2f1-ac538157c313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:06.897394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d691d00-16b3-44a8-a2f1-ac538157c313 admin admin] PATH_INFO: `/v3/projects/6f96edb40c4848b3912714f4b3d94a8a/users/71919930721743cab5cbbbce605a8b78/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:06.912813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d691d00-16b3-44a8-a2f1-ac538157c313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f96edb40c4848b3912714f4b3d94a8a, user_id=71919930721743cab5cbbbce605a8b78, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:06.915703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d691d00-16b3-44a8-a2f1-ac538157c313 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:06.989268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 981/1983] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:06 2022] PUT /identity/v3/projects/6f96edb40c4848b3912714f4b3d94a8a/users/71919930721743cab5cbbbce605a8b78/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:07.000044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0b1a8-215d-4b4d-be02-fec1b6e6d606 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:07.000165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0b1a8-215d-4b4d-be02-fec1b6e6d606 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:07.000419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0b1a8-215d-4b4d-be02-fec1b6e6d606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:07.025125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:07.033414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7be0b1a8-215d-4b4d-be02-fec1b6e6d606 None None] MFA Rules not processed for user `71919930721743cab5cbbbce605a8b78`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:07.034641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7be0b1a8-215d-4b4d-be02-fec1b6e6d606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:07.115857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1003/1984] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:07.120737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d691d00-16b3-44a8-a2f1-ac538157c313 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:07.197850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d691d00-16b3-44a8-a2f1-ac538157c313 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:07.203004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5fd2b-d5cc-4af1-b3f1-13f39fadbfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:07.206441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5fd2b-d5cc-4af1-b3f1-13f39fadbfd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:07.206979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5fd2b-d5cc-4af1-b3f1-13f39fadbfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:07.207359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5fd2b-d5cc-4af1-b3f1-13f39fadbfd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:07.208273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5fd2b-d5cc-4af1-b3f1-13f39fadbfd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:07.210229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5fd2b-d5cc-4af1-b3f1-13f39fadbfd0 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:07.286647 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 982/1985] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:07 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:07.290651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0b1a8-215d-4b4d-be02-fec1b6e6d606 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:07.301651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0b1a8-215d-4b4d-be02-fec1b6e6d606 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:07.386120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce1d2b-2458-4cde-b5e3-55d489e39054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:07.387402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1d2b-2458-4cde-b5e3-55d489e39054 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:07.387905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1d2b-2458-4cde-b5e3-55d489e39054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:07.388407 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1d2b-2458-4cde-b5e3-55d489e39054 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:07.389352 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce1d2b-2458-4cde-b5e3-55d489e39054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:07.391601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce1d2b-2458-4cde-b5e3-55d489e39054 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:07.462375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1004/1986] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:07 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:07.466367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5fd2b-d5cc-4af1-b3f1-13f39fadbfd0 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:07.477743 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5fd2b-d5cc-4af1-b3f1-13f39fadbfd0 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:07.482493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a3029-c4c3-4652-a258-d2c1ab6d2809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:07.483110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a3029-c4c3-4652-a258-d2c1ab6d2809 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:07.483395 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a3029-c4c3-4652-a258-d2c1ab6d2809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:07.483676 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a3029-c4c3-4652-a258-d2c1ab6d2809 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:07.484384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a3029-c4c3-4652-a258-d2c1ab6d2809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:07.486021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a3029-c4c3-4652-a258-d2c1ab6d2809 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:07.497859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 983/1987] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:07.502122 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce1d2b-2458-4cde-b5e3-55d489e39054 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:07.513062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce1d2b-2458-4cde-b5e3-55d489e39054 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:07.518051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148990b-45e2-4ac9-bcd2-014a0a30c4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:07.518773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148990b-45e2-4ac9-bcd2-014a0a30c4cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:07.519072 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148990b-45e2-4ac9-bcd2-014a0a30c4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:07.519349 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148990b-45e2-4ac9-bcd2-014a0a30c4cf admin admin] PATH_INFO: `/v3/projects/b2a54fc211944a7996a908ddd4126ec3/users/6829084a549e460ba7363b72a77d0043/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:07.534685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148990b-45e2-4ac9-bcd2-014a0a30c4cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a54fc211944a7996a908ddd4126ec3, user_id=6829084a549e460ba7363b72a77d0043, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:07.537918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148990b-45e2-4ac9-bcd2-014a0a30c4cf admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:07.623386 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1005/1988] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:07 2022] PUT /identity/v3/projects/b2a54fc211944a7996a908ddd4126ec3/users/6829084a549e460ba7363b72a77d0043/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:07.627491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a3029-c4c3-4652-a258-d2c1ab6d2809 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:07.639057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a3029-c4c3-4652-a258-d2c1ab6d2809 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:07.647581 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:07.698567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9541e-2af3-458d-ac92-a9540d862ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:07.699227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9541e-2af3-458d-ac92-a9540d862ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:07.699517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9541e-2af3-458d-ac92-a9540d862ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:07.699797 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9541e-2af3-458d-ac92-a9540d862ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:07.700503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9541e-2af3-458d-ac92-a9540d862ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:07.702127 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9541e-2af3-458d-ac92-a9540d862ed1 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:07.712483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 984/1989] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:07.717005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148990b-45e2-4ac9-bcd2-014a0a30c4cf admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:07.727922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148990b-45e2-4ac9-bcd2-014a0a30c4cf admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:07.737949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934a590-a07a-469c-9ebe-844d79cfbe5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:07.737949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934a590-a07a-469c-9ebe-844d79cfbe5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:07.737949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934a590-a07a-469c-9ebe-844d79cfbe5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:07.737949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934a590-a07a-469c-9ebe-844d79cfbe5e admin admin] PATH_INFO: `/v3/projects/b2a54fc211944a7996a908ddd4126ec3/users/6829084a549e460ba7363b72a77d0043/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:07.738343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934a590-a07a-469c-9ebe-844d79cfbe5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a54fc211944a7996a908ddd4126ec3, user_id=6829084a549e460ba7363b72a77d0043, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:07.738917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934a590-a07a-469c-9ebe-844d79cfbe5e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:07.750379 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1006/1990] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:07 2022] PUT /identity/v3/projects/b2a54fc211944a7996a908ddd4126ec3/users/6829084a549e460ba7363b72a77d0043/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:07.762279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547060-f3ea-4e67-8c9a-d21941ff2260 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:07.762643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547060-f3ea-4e67-8c9a-d21941ff2260 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:07.763409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef547060-f3ea-4e67-8c9a-d21941ff2260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:07.798028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-ef547060-f3ea-4e67-8c9a-d21941ff2260 None None] MFA Rules not processed for user `6829084a549e460ba7363b72a77d0043`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:07.799434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ef547060-f3ea-4e67-8c9a-d21941ff2260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:07.879368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 985/1991] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:07.891408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934a590-a07a-469c-9ebe-844d79cfbe5e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:07.893370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f934a590-a07a-469c-9ebe-844d79cfbe5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:07.955063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934a590-a07a-469c-9ebe-844d79cfbe5e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:07.958874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20ac05-e462-4a46-a362-c13a632f39b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:07.959557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ac05-e462-4a46-a362-c13a632f39b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:07.959796 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ac05-e462-4a46-a362-c13a632f39b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:07.960075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ac05-e462-4a46-a362-c13a632f39b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:07.968160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:07.971459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20ac05-e462-4a46-a362-c13a632f39b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:07.973392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20ac05-e462-4a46-a362-c13a632f39b2 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:07.984552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1007/1992] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:08.095397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef547060-f3ea-4e67-8c9a-d21941ff2260 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:08.109114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef547060-f3ea-4e67-8c9a-d21941ff2260 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:08.113015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6aa4e-b45f-413b-9c77-c8a4e0bfc8f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:08.113851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6aa4e-b45f-413b-9c77-c8a4e0bfc8f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:08.114603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6aa4e-b45f-413b-9c77-c8a4e0bfc8f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:08.114874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6aa4e-b45f-413b-9c77-c8a4e0bfc8f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:08.180168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6aa4e-b45f-413b-9c77-c8a4e0bfc8f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:08.181659 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6aa4e-b45f-413b-9c77-c8a4e0bfc8f1 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:08.191899 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 986/1993] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:11.755152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20ac05-e462-4a46-a362-c13a632f39b2 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:11.756562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1d20ac05-e462-4a46-a362-c13a632f39b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:11.825145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20ac05-e462-4a46-a362-c13a632f39b2 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:11.829769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387d580-363c-4ea9-9b0e-7a0e57ca6347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:11.830413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387d580-363c-4ea9-9b0e-7a0e57ca6347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:11.830691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387d580-363c-4ea9-9b0e-7a0e57ca6347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:11.830966 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387d580-363c-4ea9-9b0e-7a0e57ca6347 admin admin] PATH_INFO: `/v3/users/c3c41ba1c05549368d858b7932ef5e8d` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:11.832332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387d580-363c-4ea9-9b0e-7a0e57ca6347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c41ba1c05549368d858b7932ef5e8d)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:11.834273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387d580-363c-4ea9-9b0e-7a0e57ca6347 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:12.110693 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e387d580-363c-4ea9-9b0e-7a0e57ca6347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c41ba1c05549368d858b7932ef5e8d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.111569 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e387d580-363c-4ea9-9b0e-7a0e57ca6347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c41ba1c05549368d858b7932ef5e8d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.119155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e387d580-363c-4ea9-9b0e-7a0e57ca6347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c41ba1c05549368d858b7932ef5e8d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.171667 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6aa4e-b45f-413b-9c77-c8a4e0bfc8f1 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:12.173306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ddc6aa4e-b45f-413b-9c77-c8a4e0bfc8f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:12.180329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e387d580-363c-4ea9-9b0e-7a0e57ca6347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c41ba1c05549368d858b7932ef5e8d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.183315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e387d580-363c-4ea9-9b0e-7a0e57ca6347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c41ba1c05549368d858b7932ef5e8d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.187857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6aa4e-b45f-413b-9c77-c8a4e0bfc8f1 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:12.192778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:12.193649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:12.193932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:12.194212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] PATH_INFO: `/v3/users/66c90572f6bb47c39f2855933b69b37b` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:12.195418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=66c90572f6bb47c39f2855933b69b37b)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:12.197404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:12.204743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1008/1994] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:11 2022] DELETE /identity/v3/users/c3c41ba1c05549368d858b7932ef5e8d => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:12.244365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332fa51-09a0-4c7c-b6cf-b97ff7a6951f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:12.244365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332fa51-09a0-4c7c-b6cf-b97ff7a6951f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:12.244365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332fa51-09a0-4c7c-b6cf-b97ff7a6951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:12.244365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c332fa51-09a0-4c7c-b6cf-b97ff7a6951f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:12.244365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c332fa51-09a0-4c7c-b6cf-b97ff7a6951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:12.294232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1009/1995] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:12.305613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332fa51-09a0-4c7c-b6cf-b97ff7a6951f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:12.306918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c332fa51-09a0-4c7c-b6cf-b97ff7a6951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:12.314020 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:12.325310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:12.370073 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332fa51-09a0-4c7c-b6cf-b97ff7a6951f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:12.374885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc74be-832f-4c42-9655-7f027e896b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:12.376341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc74be-832f-4c42-9655-7f027e896b5d service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:12.376341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc74be-832f-4c42-9655-7f027e896b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:12.376611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc74be-832f-4c42-9655-7f027e896b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:12.399833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc74be-832f-4c42-9655-7f027e896b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:12.401471 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc74be-832f-4c42-9655-7f027e896b5d service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:12.411238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1010/1996] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:14:12.440126 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66c90572f6bb47c39f2855933b69b37b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.447545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66c90572f6bb47c39f2855933b69b37b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.524764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66c90572f6bb47c39f2855933b69b37b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.525412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66c90572f6bb47c39f2855933b69b37b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.537897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66c90572f6bb47c39f2855933b69b37b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:12.543767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 987/1997] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:12 2022] DELETE /identity/v3/users/66c90572f6bb47c39f2855933b69b37b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:12.548377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48c3cc-ddcb-4258-a11c-6219af608d8f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:12.548670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48c3cc-ddcb-4258-a11c-6219af608d8f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:12.549019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48c3cc-ddcb-4258-a11c-6219af608d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:12.567973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-fd48c3cc-ddcb-4258-a11c-6219af608d8f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:12.569255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fd48c3cc-ddcb-4258-a11c-6219af608d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:12.631367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1011/1998] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:12.642396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:12.644057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:12.708474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6622d2-6793-4c2e-9bf7-82d9a4f81eca admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:12.712367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e14537-a8ab-4e46-97d7-2b2b61a9ff98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:12.713218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e14537-a8ab-4e46-97d7-2b2b61a9ff98 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:12.713331 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e14537-a8ab-4e46-97d7-2b2b61a9ff98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:12.713593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e14537-a8ab-4e46-97d7-2b2b61a9ff98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:12.724100 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e14537-a8ab-4e46-97d7-2b2b61a9ff98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:12.725848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e14537-a8ab-4e46-97d7-2b2b61a9ff98 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:12.737142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 988/1999] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:12.848691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48c3cc-ddcb-4258-a11c-6219af608d8f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:12.848691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fd48c3cc-ddcb-4258-a11c-6219af608d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:12.875742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48c3cc-ddcb-4258-a11c-6219af608d8f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:12.883968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:12.883968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:12.883968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:12.883968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] PATH_INFO: `/v3/projects/95965f070ef34af7954d275e57a21307` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:12.885625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95965f070ef34af7954d275e57a21307)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:12.887481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:13.001307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95965f070ef34af7954d275e57a21307', 'request_id': 'req-98695d17-17c6-4de6-9ae0-90e5b5118de9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.009465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95965f070ef34af7954d275e57a21307', 'request_id': 'req-98695d17-17c6-4de6-9ae0-90e5b5118de9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.016454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95965f070ef34af7954d275e57a21307', 'request_id': 'req-98695d17-17c6-4de6-9ae0-90e5b5118de9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.076058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] The token cache is being invalidate because project 95965f070ef34af7954d275e57a21307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:13.076530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.077321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.079357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1012/2000] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:12 2022] DELETE /identity/v3/projects/95965f070ef34af7954d275e57a21307 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:13.083497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e14537-a8ab-4e46-97d7-2b2b61a9ff98 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:13.093992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e14537-a8ab-4e46-97d7-2b2b61a9ff98 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:13.159709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bf092-68dd-450f-ac31-ece701486e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:13.160422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bf092-68dd-450f-ac31-ece701486e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.160704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bf092-68dd-450f-ac31-ece701486e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.161005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bf092-68dd-450f-ac31-ece701486e08 admin admin] PATH_INFO: `/v3/users/a55e5d2cbcda4be79b24058696d72958` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.162170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bf092-68dd-450f-ac31-ece701486e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a55e5d2cbcda4be79b24058696d72958)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:13.164683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bf092-68dd-450f-ac31-ece701486e08 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:13.240073 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:13.242142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:13.255496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695d17-17c6-4de6-9ae0-90e5b5118de9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:13.260255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:13.260877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.261261 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.261656 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] PATH_INFO: `/v3/projects/9cfde5a784a84d9bb04353268470219e` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.262762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cfde5a784a84d9bb04353268470219e)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:13.264687 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:13.356266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b76bf092-68dd-450f-ac31-ece701486e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a55e5d2cbcda4be79b24058696d72958'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.357676 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:13.364068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b76bf092-68dd-450f-ac31-ece701486e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a55e5d2cbcda4be79b24058696d72958'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.381793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:13.390535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cfde5a784a84d9bb04353268470219e', 'request_id': 'req-0af8e865-505b-4f02-8757-5cfbcbc6935b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.396358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cfde5a784a84d9bb04353268470219e', 'request_id': 'req-0af8e865-505b-4f02-8757-5cfbcbc6935b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.404259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cfde5a784a84d9bb04353268470219e', 'request_id': 'req-0af8e865-505b-4f02-8757-5cfbcbc6935b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.421928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b76bf092-68dd-450f-ac31-ece701486e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a55e5d2cbcda4be79b24058696d72958'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.422618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b76bf092-68dd-450f-ac31-ece701486e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a55e5d2cbcda4be79b24058696d72958'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.451405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b76bf092-68dd-450f-ac31-ece701486e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a55e5d2cbcda4be79b24058696d72958'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.459662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 989/2001] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:13 2022] DELETE /identity/v3/users/a55e5d2cbcda4be79b24058696d72958 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:13.499102 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] The token cache is being invalidate because project 9cfde5a784a84d9bb04353268470219e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:13.499412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.500104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:13.502086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1013/2002] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:13 2022] DELETE /identity/v3/projects/9cfde5a784a84d9bb04353268470219e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:13.507032 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb42c2d-060e-4d05-835a-7cfc14d60dff None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.507418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb42c2d-060e-4d05-835a-7cfc14d60dff None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.507703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb42c2d-060e-4d05-835a-7cfc14d60dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.528709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6fb42c2d-060e-4d05-835a-7cfc14d60dff None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:13.530009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6fb42c2d-060e-4d05-835a-7cfc14d60dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:13.595370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:13.602222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 990/2003] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:13.603555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:13.604777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e72ddd-db60-4790-8f0b-a85d3b44cc79 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.605189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e72ddd-db60-4790-8f0b-a85d3b44cc79 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.605480 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e72ddd-db60-4790-8f0b-a85d3b44cc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.631501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-16e72ddd-db60-4790-8f0b-a85d3b44cc79 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:13.635090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-16e72ddd-db60-4790-8f0b-a85d3b44cc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:13.650591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 991/2004] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:13.651932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e72ddd-db60-4790-8f0b-a85d3b44cc79 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:13.664831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e72ddd-db60-4790-8f0b-a85d3b44cc79 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:13.672034 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1cc00-339f-4d61-803a-399c56e0875d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:13.672762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1cc00-339f-4d61-803a-399c56e0875d admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.673178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1cc00-339f-4d61-803a-399c56e0875d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.673533 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1cc00-339f-4d61-803a-399c56e0875d admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.674338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b1cc00-339f-4d61-803a-399c56e0875d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:13.674719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b1cc00-339f-4d61-803a-399c56e0875d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:13.676722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b1cc00-339f-4d61-803a-399c56e0875d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:13.703448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 992/2005] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:13.707644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1cc00-339f-4d61-803a-399c56e0875d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:13.711084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8e865-505b-4f02-8757-5cfbcbc6935b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:13.718843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1cc00-339f-4d61-803a-399c56e0875d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:13.719023 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:13.719556 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.719842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.720121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] PATH_INFO: `/v3/users/a59b8216aa214553a442c1f897fd7526` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.721365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a59b8216aa214553a442c1f897fd7526)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:13.723323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:13.724406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39ade0-f25d-4eb1-a593-78ab20614aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:13.725140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39ade0-f25d-4eb1-a593-78ab20614aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.725498 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39ade0-f25d-4eb1-a593-78ab20614aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.725857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39ade0-f25d-4eb1-a593-78ab20614aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.727286 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39ade0-f25d-4eb1-a593-78ab20614aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:13.729370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39ade0-f25d-4eb1-a593-78ab20614aaf admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:13.843938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 993/2006] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:13 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:13.844941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39ade0-f25d-4eb1-a593-78ab20614aaf admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:13.908262 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39ade0-f25d-4eb1-a593-78ab20614aaf admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:13.913374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76465b-6c5a-44ab-9de5-1c22425c1008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:13.914001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76465b-6c5a-44ab-9de5-1c22425c1008 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:13.914791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76465b-6c5a-44ab-9de5-1c22425c1008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:13.922864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76465b-6c5a-44ab-9de5-1c22425c1008 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:13.922864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76465b-6c5a-44ab-9de5-1c22425c1008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:13.922864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76465b-6c5a-44ab-9de5-1c22425c1008 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.033149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 994/2007] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:13 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:14.034066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76465b-6c5a-44ab-9de5-1c22425c1008 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.035664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ea76465b-6c5a-44ab-9de5-1c22425c1008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:14.048721 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76465b-6c5a-44ab-9de5-1c22425c1008 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.049226 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a59b8216aa214553a442c1f897fd7526'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:14.050182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a59b8216aa214553a442c1f897fd7526'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:14.054174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.055558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.055847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.056755 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] PATH_INFO: `/v3/projects/daa113f206fb4715ac8918f76a385ba9` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.057639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa113f206fb4715ac8918f76a385ba9)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.059814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.066372 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a59b8216aa214553a442c1f897fd7526'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:14.071090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:14.129736 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a59b8216aa214553a442c1f897fd7526'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:14.130392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a59b8216aa214553a442c1f897fd7526'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:14.149099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1014/2008] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:13 2022] DELETE /identity/v3/users/a59b8216aa214553a442c1f897fd7526 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:14.150026 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.151399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:14.173725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa113f206fb4715ac8918f76a385ba9', 'request_id': 'req-9deb514c-5506-48df-8033-a159f74210a2'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:14.179296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa113f206fb4715ac8918f76a385ba9', 'request_id': 'req-9deb514c-5506-48df-8033-a159f74210a2'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:14.201442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:14.219412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b881b-4b82-4b6e-b910-404086a1b47c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.238099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa113f206fb4715ac8918f76a385ba9', 'request_id': 'req-9deb514c-5506-48df-8033-a159f74210a2'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:14.250131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] The token cache is being invalidate because project daa113f206fb4715ac8918f76a385ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:14.250612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:14.251233 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9deb514c-5506-48df-8033-a159f74210a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:14.252802 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 995/2009] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:14 2022] DELETE /identity/v3/projects/daa113f206fb4715ac8918f76a385ba9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:14.254462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5acf4f-96b1-43eb-a2ef-efa8535d3d82 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.254746 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5acf4f-96b1-43eb-a2ef-efa8535d3d82 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.255019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5acf4f-96b1-43eb-a2ef-efa8535d3d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.274051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-be5acf4f-96b1-43eb-a2ef-efa8535d3d82 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:14.275269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-be5acf4f-96b1-43eb-a2ef-efa8535d3d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:14.277409 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12f07e-6550-47dd-bfa0-1b0335986249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.279465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12f07e-6550-47dd-bfa0-1b0335986249 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.279465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12f07e-6550-47dd-bfa0-1b0335986249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.279465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12f07e-6550-47dd-bfa0-1b0335986249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.279465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12f07e-6550-47dd-bfa0-1b0335986249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.281032 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12f07e-6550-47dd-bfa0-1b0335986249 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.283093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 996/2010] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:14.284287 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db998d-aab9-4efe-bee2-f16c22ba2c4f None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.284574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db998d-aab9-4efe-bee2-f16c22ba2c4f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.284854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db998d-aab9-4efe-bee2-f16c22ba2c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.293692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1015/2011] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:14.294546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12f07e-6550-47dd-bfa0-1b0335986249 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.296186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2d12f07e-6550-47dd-bfa0-1b0335986249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:14.305138 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-05db998d-aab9-4efe-bee2-f16c22ba2c4f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:14.306394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-05db998d-aab9-4efe-bee2-f16c22ba2c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:14.314681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 997/2012] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:14.315096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db998d-aab9-4efe-bee2-f16c22ba2c4f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.327551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-05db998d-aab9-4efe-bee2-f16c22ba2c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:14.327660 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db998d-aab9-4efe-bee2-f16c22ba2c4f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.332733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f09d2-a28b-426d-ae85-a531f9c703d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.333302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f09d2-a28b-426d-ae85-a531f9c703d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.333603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f09d2-a28b-426d-ae85-a531f9c703d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.333861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f09d2-a28b-426d-ae85-a531f9c703d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.334569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f09d2-a28b-426d-ae85-a531f9c703d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.336710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f09d2-a28b-426d-ae85-a531f9c703d9 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.347507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 998/2013] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:14.348609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b046fb4-5fbb-47dd-830b-e8cead09f4ec None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.348935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b046fb4-5fbb-47dd-830b-e8cead09f4ec None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.349210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b046fb4-5fbb-47dd-830b-e8cead09f4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.356021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12f07e-6550-47dd-bfa0-1b0335986249 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.359700 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d42ffa-e6b3-4370-b5d0-0f25765bec52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.360321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d42ffa-e6b3-4370-b5d0-0f25765bec52 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.360605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d42ffa-e6b3-4370-b5d0-0f25765bec52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.360886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d42ffa-e6b3-4370-b5d0-0f25765bec52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.370287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d42ffa-e6b3-4370-b5d0-0f25765bec52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.370504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5b046fb4-5fbb-47dd-830b-e8cead09f4ec None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:14.371754 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d42ffa-e6b3-4370-b5d0-0f25765bec52 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.371870 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5b046fb4-5fbb-47dd-830b-e8cead09f4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:14.383699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 999/2014] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:14.383699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b046fb4-5fbb-47dd-830b-e8cead09f4ec None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.383928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1016/2015] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:14.385157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d42ffa-e6b3-4370-b5d0-0f25765bec52 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.391157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b046fb4-5fbb-47dd-830b-e8cead09f4ec None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.396994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab4513-5b96-423d-b502-487531a58d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.397252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d42ffa-e6b3-4370-b5d0-0f25765bec52 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.397747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab4513-5b96-423d-b502-487531a58d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.398035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab4513-5b96-423d-b502-487531a58d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.398325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab4513-5b96-423d-b502-487531a58d6d admin admin] PATH_INFO: `/v3/projects/8155dc06826e4156b281dc6f0e6ccda6/users/7e91fbe205fc4f0a89ba597c29081a7d/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.403722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85030974-8fea-4a8c-b0b2-bc54778eed34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.404553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85030974-8fea-4a8c-b0b2-bc54778eed34 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.404912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85030974-8fea-4a8c-b0b2-bc54778eed34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.405307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85030974-8fea-4a8c-b0b2-bc54778eed34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.406094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85030974-8fea-4a8c-b0b2-bc54778eed34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:14.406480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85030974-8fea-4a8c-b0b2-bc54778eed34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.408177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85030974-8fea-4a8c-b0b2-bc54778eed34 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.431294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1017/2016] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:14.431294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85030974-8fea-4a8c-b0b2-bc54778eed34 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.432239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab4513-5b96-423d-b502-487531a58d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8155dc06826e4156b281dc6f0e6ccda6, user_id=7e91fbe205fc4f0a89ba597c29081a7d, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.435686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab4513-5b96-423d-b502-487531a58d6d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.439253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85030974-8fea-4a8c-b0b2-bc54778eed34 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.444626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377440e-d8c1-457b-90bf-f67a80e51c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.445297 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377440e-d8c1-457b-90bf-f67a80e51c87 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.445854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377440e-d8c1-457b-90bf-f67a80e51c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.446140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377440e-d8c1-457b-90bf-f67a80e51c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.446826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377440e-d8c1-457b-90bf-f67a80e51c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.449005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377440e-d8c1-457b-90bf-f67a80e51c87 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.531189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1000/2017] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:14 2022] PUT /identity/v3/projects/8155dc06826e4156b281dc6f0e6ccda6/users/7e91fbe205fc4f0a89ba597c29081a7d/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:14.546618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24b06-20c0-4acf-b8b6-2a9f1f8fbd5d None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.546904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24b06-20c0-4acf-b8b6-2a9f1f8fbd5d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.547207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24b06-20c0-4acf-b8b6-2a9f1f8fbd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.567693 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1018/2018] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:14 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:14.571877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377440e-d8c1-457b-90bf-f67a80e51c87 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.601636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9bc24b06-20c0-4acf-b8b6-2a9f1f8fbd5d None None] MFA Rules not processed for user `7e91fbe205fc4f0a89ba597c29081a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:14.603121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9bc24b06-20c0-4acf-b8b6-2a9f1f8fbd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:14.661924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:14.667177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377440e-d8c1-457b-90bf-f67a80e51c87 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.674883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642255a-80f5-4565-b574-a5b3b807f155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.674883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642255a-80f5-4565-b574-a5b3b807f155 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.674883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642255a-80f5-4565-b574-a5b3b807f155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.675538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642255a-80f5-4565-b574-a5b3b807f155 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.675834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642255a-80f5-4565-b574-a5b3b807f155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.678247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642255a-80f5-4565-b574-a5b3b807f155 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.693678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1001/2019] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:14.707744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc24b06-20c0-4acf-b8b6-2a9f1f8fbd5d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.717011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9bc24b06-20c0-4acf-b8b6-2a9f1f8fbd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:14.762151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:14.788299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc24b06-20c0-4acf-b8b6-2a9f1f8fbd5d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.793583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efe4b8-4388-483e-b1b7-a0f8f90e2aed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.794223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efe4b8-4388-483e-b1b7-a0f8f90e2aed service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.797847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efe4b8-4388-483e-b1b7-a0f8f90e2aed service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.798371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efe4b8-4388-483e-b1b7-a0f8f90e2aed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.800792 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1019/2020] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:14 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:14.804783 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642255a-80f5-4565-b574-a5b3b807f155 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.809197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efe4b8-4388-483e-b1b7-a0f8f90e2aed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.810777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efe4b8-4388-483e-b1b7-a0f8f90e2aed service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.826122 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642255a-80f5-4565-b574-a5b3b807f155 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.832759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73993dca-5773-462a-8440-e2dd41d85dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.834023 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73993dca-5773-462a-8440-e2dd41d85dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.834311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73993dca-5773-462a-8440-e2dd41d85dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.834577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1002/2021] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:14.834716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73993dca-5773-462a-8440-e2dd41d85dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.835334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73993dca-5773-462a-8440-e2dd41d85dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.838831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73993dca-5773-462a-8440-e2dd41d85dbe admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.850256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1020/2022] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:14:14.854449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efe4b8-4388-483e-b1b7-a0f8f90e2aed service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.865563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efe4b8-4388-483e-b1b7-a0f8f90e2aed service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.871066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01664d5-0330-40e5-98e9-581757bf3cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.871684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01664d5-0330-40e5-98e9-581757bf3cde admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.871970 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01664d5-0330-40e5-98e9-581757bf3cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.872247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01664d5-0330-40e5-98e9-581757bf3cde admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.872954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01664d5-0330-40e5-98e9-581757bf3cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.874550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01664d5-0330-40e5-98e9-581757bf3cde admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.885497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1003/2023] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:14.889746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73993dca-5773-462a-8440-e2dd41d85dbe admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.900691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73993dca-5773-462a-8440-e2dd41d85dbe admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.905453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c200d-407e-49a1-8edf-2f8473574778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.906133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c200d-407e-49a1-8edf-2f8473574778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.906419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c200d-407e-49a1-8edf-2f8473574778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.906947 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c200d-407e-49a1-8edf-2f8473574778 admin admin] PATH_INFO: `/v3/projects/ba7ca9875a34453cb582aba7770ba4b6/users/168247d50fd04c08907841280bb8ec6b/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.921885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c200d-407e-49a1-8edf-2f8473574778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba7ca9875a34453cb582aba7770ba4b6, user_id=168247d50fd04c08907841280bb8ec6b, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.924549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c200d-407e-49a1-8edf-2f8473574778 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:14.950310 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01664d5-0330-40e5-98e9-581757bf3cde admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:14.951650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e01664d5-0330-40e5-98e9-581757bf3cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:14.973939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01664d5-0330-40e5-98e9-581757bf3cde admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:14.979711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:14.980439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:14.980691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:14.981036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] PATH_INFO: `/v3/projects/d39054ef8f67447d8f19ba23505d1b37` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:14.982288 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d39054ef8f67447d8f19ba23505d1b37)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:14.984185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:15.027491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1021/2024] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:14 2022] PUT /identity/v3/projects/ba7ca9875a34453cb582aba7770ba4b6/users/168247d50fd04c08907841280bb8ec6b/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:15.037719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cad05d-82b5-4dcd-ba3e-23a336879385 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:15.038010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cad05d-82b5-4dcd-ba3e-23a336879385 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:15.038292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cad05d-82b5-4dcd-ba3e-23a336879385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:15.067022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f6cad05d-82b5-4dcd-ba3e-23a336879385 None None] MFA Rules not processed for user `168247d50fd04c08907841280bb8ec6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:15.068277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f6cad05d-82b5-4dcd-ba3e-23a336879385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:15.135880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1022/2025] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:15.141163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cad05d-82b5-4dcd-ba3e-23a336879385 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:15.149366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd39054ef8f67447d8f19ba23505d1b37', 'request_id': 'req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:15.155258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd39054ef8f67447d8f19ba23505d1b37', 'request_id': 'req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:15.208762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cad05d-82b5-4dcd-ba3e-23a336879385 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:15.215741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd39054ef8f67447d8f19ba23505d1b37', 'request_id': 'req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:15.230499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] The token cache is being invalidate because project d39054ef8f67447d8f19ba23505d1b37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:15.231130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:15.231822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:15.233510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1004/2026] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:14 2022] DELETE /identity/v3/projects/d39054ef8f67447d8f19ba23505d1b37 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:15.237463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:15.238955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:15.276415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54988a-6586-45f7-a4ce-016b303e0737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:15.277649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54988a-6586-45f7-a4ce-016b303e0737 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:15.277767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54988a-6586-45f7-a4ce-016b303e0737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:15.278216 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54988a-6586-45f7-a4ce-016b303e0737 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:15.279458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54988a-6586-45f7-a4ce-016b303e0737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:15.281433 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54988a-6586-45f7-a4ce-016b303e0737 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:15.318045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7790bd0-c018-400a-93b3-e0c7c1d5abb6 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:15.322871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:15.325586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:15.325586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:15.325586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] PATH_INFO: `/v3/users/71919930721743cab5cbbbce605a8b78` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:15.325586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71919930721743cab5cbbbce605a8b78)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:15.327284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:15.384508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1023/2027] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:15 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:15.385552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54988a-6586-45f7-a4ce-016b303e0737 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:15.387173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8b54988a-6586-45f7-a4ce-016b303e0737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:15.455884 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54988a-6586-45f7-a4ce-016b303e0737 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:15.460649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b69dd-b1da-46e7-86be-b9071198eff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:15.461315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b69dd-b1da-46e7-86be-b9071198eff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:15.461603 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b69dd-b1da-46e7-86be-b9071198eff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:15.461886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b69dd-b1da-46e7-86be-b9071198eff5 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:15.462558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b69dd-b1da-46e7-86be-b9071198eff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:15.464471 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b69dd-b1da-46e7-86be-b9071198eff5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:15.562778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1024/2028] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:15 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:15.566891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b69dd-b1da-46e7-86be-b9071198eff5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:15.576997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71919930721743cab5cbbbce605a8b78'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:15.577598 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b69dd-b1da-46e7-86be-b9071198eff5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:15.583615 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:15.588649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71919930721743cab5cbbbce605a8b78'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:15.593008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ec94a-3d7a-4d83-ada9-20e25f3f0b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:15.593644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ec94a-3d7a-4d83-ada9-20e25f3f0b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:15.593927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ec94a-3d7a-4d83-ada9-20e25f3f0b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:15.594201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ec94a-3d7a-4d83-ada9-20e25f3f0b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:15.598712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ec94a-3d7a-4d83-ada9-20e25f3f0b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:15.598712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ec94a-3d7a-4d83-ada9-20e25f3f0b2c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:15.610058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1025/2029] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:14:15.619260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ec94a-3d7a-4d83-ada9-20e25f3f0b2c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:15.629500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ec94a-3d7a-4d83-ada9-20e25f3f0b2c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:15.634324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319e58e-2a87-4528-95bf-c0777f546621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:15.634975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319e58e-2a87-4528-95bf-c0777f546621 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:15.635274 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319e58e-2a87-4528-95bf-c0777f546621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:15.635664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319e58e-2a87-4528-95bf-c0777f546621 admin admin] PATH_INFO: `/v3/projects/02b2b143dff740fea29abd444e3a0d52/users/9bf87dc8311e4596b6203fd37294ace9/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:15.650985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319e58e-2a87-4528-95bf-c0777f546621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b2b143dff740fea29abd444e3a0d52, user_id=9bf87dc8311e4596b6203fd37294ace9, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:15.655061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319e58e-2a87-4528-95bf-c0777f546621 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:15.672637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71919930721743cab5cbbbce605a8b78'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:15.673310 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71919930721743cab5cbbbce605a8b78'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:15.687409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71919930721743cab5cbbbce605a8b78'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:15.697071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1005/2030] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:15 2022] DELETE /identity/v3/users/71919930721743cab5cbbbce605a8b78 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:15.740554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1026/2031] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:15 2022] PUT /identity/v3/projects/02b2b143dff740fea29abd444e3a0d52/users/9bf87dc8311e4596b6203fd37294ace9/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:15.740873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:15.742986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:15.817922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad6c09-4e89-49d0-8c00-657087cb0d74 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:15.878041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a7cb7-f8f1-4881-bfeb-78494a9b11fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:15.878673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a7cb7-f8f1-4881-bfeb-78494a9b11fe admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:15.878955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a7cb7-f8f1-4881-bfeb-78494a9b11fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:15.879230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a7cb7-f8f1-4881-bfeb-78494a9b11fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:15.879941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a7cb7-f8f1-4881-bfeb-78494a9b11fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:15.881700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a7cb7-f8f1-4881-bfeb-78494a9b11fe admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:15.894543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1006/2032] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:15.899511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319e58e-2a87-4528-95bf-c0777f546621 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:15.910797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319e58e-2a87-4528-95bf-c0777f546621 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:15.916796 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b572a6-2ddc-4ee5-8bea-c2ea26829336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:15.917563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b572a6-2ddc-4ee5-8bea-c2ea26829336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:15.917846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b572a6-2ddc-4ee5-8bea-c2ea26829336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:15.918125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b572a6-2ddc-4ee5-8bea-c2ea26829336 admin admin] PATH_INFO: `/v3/projects/02b2b143dff740fea29abd444e3a0d52/users/9bf87dc8311e4596b6203fd37294ace9/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:15.920036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b572a6-2ddc-4ee5-8bea-c2ea26829336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b2b143dff740fea29abd444e3a0d52, user_id=9bf87dc8311e4596b6203fd37294ace9, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:15.922939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b572a6-2ddc-4ee5-8bea-c2ea26829336 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:15.938223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1027/2033] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:15 2022] PUT /identity/v3/projects/02b2b143dff740fea29abd444e3a0d52/users/9bf87dc8311e4596b6203fd37294ace9/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:15.948349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db3bad-50b0-4f52-bfcc-cdb319c26b7c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:15.948622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db3bad-50b0-4f52-bfcc-cdb319c26b7c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:15.948899 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db3bad-50b0-4f52-bfcc-cdb319c26b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:15.983348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d6db3bad-50b0-4f52-bfcc-cdb319c26b7c None None] MFA Rules not processed for user `9bf87dc8311e4596b6203fd37294ace9`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:15.985613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d6db3bad-50b0-4f52-bfcc-cdb319c26b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:16.067558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1007/2034] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:16.087494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b572a6-2ddc-4ee5-8bea-c2ea26829336 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:16.090703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e2b572a6-2ddc-4ee5-8bea-c2ea26829336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:16.160557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b572a6-2ddc-4ee5-8bea-c2ea26829336 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:16.164281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593270d6-a4a2-4822-8ab2-3fefd1cc4f87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:16.164908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593270d6-a4a2-4822-8ab2-3fefd1cc4f87 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:16.165222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593270d6-a4a2-4822-8ab2-3fefd1cc4f87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:16.165502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593270d6-a4a2-4822-8ab2-3fefd1cc4f87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:16.175267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593270d6-a4a2-4822-8ab2-3fefd1cc4f87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:16.176726 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593270d6-a4a2-4822-8ab2-3fefd1cc4f87 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:16.188663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1028/2035] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:16.264446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db3bad-50b0-4f52-bfcc-cdb319c26b7c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:16.265933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d6db3bad-50b0-4f52-bfcc-cdb319c26b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:16.349804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db3bad-50b0-4f52-bfcc-cdb319c26b7c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:16.354684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:16.355339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:16.355624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:16.355901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] PATH_INFO: `/v3/projects/6f96edb40c4848b3912714f4b3d94a8a` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:16.357118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f96edb40c4848b3912714f4b3d94a8a)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:16.359293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:16.474348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593270d6-a4a2-4822-8ab2-3fefd1cc4f87 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:16.483982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593270d6-a4a2-4822-8ab2-3fefd1cc4f87 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:16.494074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f96edb40c4848b3912714f4b3d94a8a', 'request_id': 'req-403bcd9a-12db-4928-87a3-38493d047a8e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:16.499942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f96edb40c4848b3912714f4b3d94a8a', 'request_id': 'req-403bcd9a-12db-4928-87a3-38493d047a8e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:16.544500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f706f2-49ad-479b-aa46-47be5c8ef76b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:16.545196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f706f2-49ad-479b-aa46-47be5c8ef76b service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:16.545608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f706f2-49ad-479b-aa46-47be5c8ef76b service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:16.545921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f706f2-49ad-479b-aa46-47be5c8ef76b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:16.547046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f96edb40c4848b3912714f4b3d94a8a', 'request_id': 'req-403bcd9a-12db-4928-87a3-38493d047a8e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:16.547726 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-27f706f2-49ad-479b-aa46-47be5c8ef76b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:16.561291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] The token cache is being invalidate because project 6f96edb40c4848b3912714f4b3d94a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:16.561745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:16.562481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:16.564412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1008/2036] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:16 2022] DELETE /identity/v3/projects/6f96edb40c4848b3912714f4b3d94a8a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:16.567979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:16.569399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:16.604922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:16.613182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f706f2-49ad-479b-aa46-47be5c8ef76b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:16.614676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f706f2-49ad-479b-aa46-47be5c8ef76b service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:16.626332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1029/2037] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:16.644886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403bcd9a-12db-4928-87a3-38493d047a8e admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:16.649466 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bc977-c55d-4b40-90e4-ef49fbbbef20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:16.650182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc977-c55d-4b40-90e4-ef49fbbbef20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:16.650433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc977-c55d-4b40-90e4-ef49fbbbef20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:16.650839 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc977-c55d-4b40-90e4-ef49fbbbef20 admin admin] PATH_INFO: `/v3/users/6829084a549e460ba7363b72a77d0043` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:16.651945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bc977-c55d-4b40-90e4-ef49fbbbef20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6829084a549e460ba7363b72a77d0043)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:16.654480 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bc977-c55d-4b40-90e4-ef49fbbbef20 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:16.936210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:16.951165 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-814bc977-c55d-4b40-90e4-ef49fbbbef20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6829084a549e460ba7363b72a77d0043'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:16.958049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-814bc977-c55d-4b40-90e4-ef49fbbbef20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6829084a549e460ba7363b72a77d0043'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:17.035961 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-814bc977-c55d-4b40-90e4-ef49fbbbef20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6829084a549e460ba7363b72a77d0043'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:17.037387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-814bc977-c55d-4b40-90e4-ef49fbbbef20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6829084a549e460ba7363b72a77d0043'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:17.054472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-814bc977-c55d-4b40-90e4-ef49fbbbef20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6829084a549e460ba7363b72a77d0043'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:17.061231 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1009/2038] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:16 2022] DELETE /identity/v3/users/6829084a549e460ba7363b72a77d0043 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:17.577620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f706f2-49ad-479b-aa46-47be5c8ef76b service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:17.595174 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f706f2-49ad-479b-aa46-47be5c8ef76b service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:17.653503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:17.655462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:17.655807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:17.656090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] PATH_INFO: `/v3/projects/b2a54fc211944a7996a908ddd4126ec3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:17.657304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a54fc211944a7996a908ddd4126ec3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:17.659341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:17.811764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a54fc211944a7996a908ddd4126ec3', 'request_id': 'req-23c7728e-9b30-476f-80d1-fdf86f6e78d4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:17.817287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a54fc211944a7996a908ddd4126ec3', 'request_id': 'req-23c7728e-9b30-476f-80d1-fdf86f6e78d4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:17.824084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a54fc211944a7996a908ddd4126ec3', 'request_id': 'req-23c7728e-9b30-476f-80d1-fdf86f6e78d4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:17.878722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] The token cache is being invalidate because project b2a54fc211944a7996a908ddd4126ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:17.879020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:17.879675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23c7728e-9b30-476f-80d1-fdf86f6e78d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:17.881265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1030/2039] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:17 2022] DELETE /identity/v3/projects/b2a54fc211944a7996a908ddd4126ec3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:17.887897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa329-1fbd-4eec-ac8c-e4f030e6e2f2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:17.888373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa329-1fbd-4eec-ac8c-e4f030e6e2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:17.888484 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa329-1fbd-4eec-ac8c-e4f030e6e2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:17.908605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8c9aa329-1fbd-4eec-ac8c-e4f030e6e2f2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:17.909859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8c9aa329-1fbd-4eec-ac8c-e4f030e6e2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:17.970307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1010/2040] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:17.978833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67839bb-2ac0-4343-adaf-92f8a32d401a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:17.979126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67839bb-2ac0-4343-adaf-92f8a32d401a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:17.979405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67839bb-2ac0-4343-adaf-92f8a32d401a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:17.999454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a67839bb-2ac0-4343-adaf-92f8a32d401a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:18.000696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a67839bb-2ac0-4343-adaf-92f8a32d401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:18.009416 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1031/2041] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:18.013739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa329-1fbd-4eec-ac8c-e4f030e6e2f2 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:18.023404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa329-1fbd-4eec-ac8c-e4f030e6e2f2 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:18.028051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df74b1-dd7d-4232-80c3-c4e577a11b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:18.028656 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df74b1-dd7d-4232-80c3-c4e577a11b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:18.028940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df74b1-dd7d-4232-80c3-c4e577a11b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:18.029247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df74b1-dd7d-4232-80c3-c4e577a11b3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:18.033578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df74b1-dd7d-4232-80c3-c4e577a11b3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:18.033578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df74b1-dd7d-4232-80c3-c4e577a11b3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:18.033578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df74b1-dd7d-4232-80c3-c4e577a11b3b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:18.043250 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1011/2042] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:14:18.047811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67839bb-2ac0-4343-adaf-92f8a32d401a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:18.053800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:18.059339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67839bb-2ac0-4343-adaf-92f8a32d401a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:18.064146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c5f0c-6391-4921-9dda-fd1e921e07ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:18.064773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c5f0c-6391-4921-9dda-fd1e921e07ad admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:18.065074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c5f0c-6391-4921-9dda-fd1e921e07ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:18.065360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c5f0c-6391-4921-9dda-fd1e921e07ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:18.066049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c5f0c-6391-4921-9dda-fd1e921e07ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:18.067988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c5f0c-6391-4921-9dda-fd1e921e07ad admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:18.145849 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:18.176874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1032/2043] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:18 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:18.182146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df74b1-dd7d-4232-80c3-c4e577a11b3b admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:18.193513 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df74b1-dd7d-4232-80c3-c4e577a11b3b admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:18.270339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abd770-6bb8-440f-bc8f-98131869098c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:18.275128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abd770-6bb8-440f-bc8f-98131869098c admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:18.275483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abd770-6bb8-440f-bc8f-98131869098c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:18.275798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abd770-6bb8-440f-bc8f-98131869098c admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:18.276899 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abd770-6bb8-440f-bc8f-98131869098c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:18.279031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abd770-6bb8-440f-bc8f-98131869098c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:18.366543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1012/2044] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:18 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:18.368404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c5f0c-6391-4921-9dda-fd1e921e07ad admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:18.378806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c5f0c-6391-4921-9dda-fd1e921e07ad admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:18.384685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e71d97-f601-464c-ad59-c212de84c10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:18.385339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e71d97-f601-464c-ad59-c212de84c10b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:18.385625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e71d97-f601-464c-ad59-c212de84c10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:18.385902 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e71d97-f601-464c-ad59-c212de84c10b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:18.386607 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e71d97-f601-464c-ad59-c212de84c10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:18.388205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e71d97-f601-464c-ad59-c212de84c10b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:18.398217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1033/2045] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:18.402218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abd770-6bb8-440f-bc8f-98131869098c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:18.412350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abd770-6bb8-440f-bc8f-98131869098c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:18.418002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7c753-c812-473e-9a17-e9de4f227b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:18.418620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7c753-c812-473e-9a17-e9de4f227b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:18.418925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7c753-c812-473e-9a17-e9de4f227b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:18.419180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7c753-c812-473e-9a17-e9de4f227b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:18.423063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7c753-c812-473e-9a17-e9de4f227b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:18.423063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7c753-c812-473e-9a17-e9de4f227b8a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:18.433898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1013/2046] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:18.438161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e71d97-f601-464c-ad59-c212de84c10b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:18.451942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e71d97-f601-464c-ad59-c212de84c10b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:18.453451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7256f1-5cae-4c3f-8bc3-a32233d3ea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:18.454120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7256f1-5cae-4c3f-8bc3-a32233d3ea8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:18.454408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7256f1-5cae-4c3f-8bc3-a32233d3ea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:18.454688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7256f1-5cae-4c3f-8bc3-a32233d3ea8c admin admin] PATH_INFO: `/v3/projects/b1a81d39608a400a8ba5c808c1559700/users/6b8c2c4bdd614aa9b9f8cb286ed5ad55/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:18.469780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7256f1-5cae-4c3f-8bc3-a32233d3ea8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a81d39608a400a8ba5c808c1559700, user_id=6b8c2c4bdd614aa9b9f8cb286ed5ad55, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:18.472462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7256f1-5cae-4c3f-8bc3-a32233d3ea8c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:18.575832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1034/2047] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:18 2022] PUT /identity/v3/projects/b1a81d39608a400a8ba5c808c1559700/users/6b8c2c4bdd614aa9b9f8cb286ed5ad55/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:18.587698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2f683-8a8d-4ea2-be46-d2597105be5c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:18.587987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2f683-8a8d-4ea2-be46-d2597105be5c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:18.588294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2f683-8a8d-4ea2-be46-d2597105be5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:18.619818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-87e2f683-8a8d-4ea2-be46-d2597105be5c None None] MFA Rules not processed for user `6b8c2c4bdd614aa9b9f8cb286ed5ad55`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:18.621341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-87e2f683-8a8d-4ea2-be46-d2597105be5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:18.685878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1014/2048] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:18.690665 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7256f1-5cae-4c3f-8bc3-a32233d3ea8c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:18.757968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7256f1-5cae-4c3f-8bc3-a32233d3ea8c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:18.762843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451d198-21fb-47fe-8518-b2dfc2e2944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:18.763451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451d198-21fb-47fe-8518-b2dfc2e2944b admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:18.763738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451d198-21fb-47fe-8518-b2dfc2e2944b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:18.764026 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451d198-21fb-47fe-8518-b2dfc2e2944b admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:18.764706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451d198-21fb-47fe-8518-b2dfc2e2944b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:18.766670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451d198-21fb-47fe-8518-b2dfc2e2944b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:18.932046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1035/2049] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:18 2022] POST /identity/v3/projects => generated 375 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:18.936308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2f683-8a8d-4ea2-be46-d2597105be5c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:18.947727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2f683-8a8d-4ea2-be46-d2597105be5c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:19.003834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6043ab-fb47-4632-a90a-cc1fa05e59d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:19.004441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6043ab-fb47-4632-a90a-cc1fa05e59d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:19.004723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6043ab-fb47-4632-a90a-cc1fa05e59d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:19.005019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6043ab-fb47-4632-a90a-cc1fa05e59d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:19.005689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6043ab-fb47-4632-a90a-cc1fa05e59d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:19.007712 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6043ab-fb47-4632-a90a-cc1fa05e59d0 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:19.112851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1015/2050] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:18 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:19.116918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451d198-21fb-47fe-8518-b2dfc2e2944b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:19.132429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451d198-21fb-47fe-8518-b2dfc2e2944b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:19.134178 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97d062-2a09-444b-b4c4-ea459880744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:19.134926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97d062-2a09-444b-b4c4-ea459880744a admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:19.135282 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97d062-2a09-444b-b4c4-ea459880744a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:19.135571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97d062-2a09-444b-b4c4-ea459880744a admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:19.136286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97d062-2a09-444b-b4c4-ea459880744a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:19.138123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97d062-2a09-444b-b4c4-ea459880744a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:19.149598 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1036/2051] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:19.153718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6043ab-fb47-4632-a90a-cc1fa05e59d0 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:19.165429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6043ab-fb47-4632-a90a-cc1fa05e59d0 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:19.170367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3ac5-901f-420f-a592-1509e04512ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:19.171046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3ac5-901f-420f-a592-1509e04512ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:19.171333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3ac5-901f-420f-a592-1509e04512ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:19.171616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3ac5-901f-420f-a592-1509e04512ba admin admin] PATH_INFO: `/v3/projects/64ba5f531af540d9940802e61aca4f6e/users/341af8d25c084be5aa82f23f33933d92/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:19.190828 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb3ac5-901f-420f-a592-1509e04512ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ba5f531af540d9940802e61aca4f6e, user_id=341af8d25c084be5aa82f23f33933d92, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:19.193522 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb3ac5-901f-420f-a592-1509e04512ba admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:19.286192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1016/2052] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:19 2022] PUT /identity/v3/projects/64ba5f531af540d9940802e61aca4f6e/users/341af8d25c084be5aa82f23f33933d92/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:19.291098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97d062-2a09-444b-b4c4-ea459880744a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:19.301671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97d062-2a09-444b-b4c4-ea459880744a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:19.354133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:19.362564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f349ef5-fac9-4426-8f57-cff7e3c65688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:19.363229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349ef5-fac9-4426-8f57-cff7e3c65688 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:19.363515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349ef5-fac9-4426-8f57-cff7e3c65688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:19.363792 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349ef5-fac9-4426-8f57-cff7e3c65688 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:19.364502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f349ef5-fac9-4426-8f57-cff7e3c65688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:19.366142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f349ef5-fac9-4426-8f57-cff7e3c65688 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:19.376690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1037/2053] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:19.380884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3ac5-901f-420f-a592-1509e04512ba admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:19.391941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3ac5-901f-420f-a592-1509e04512ba admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:19.396888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf8a4e-742c-426e-b467-ce12bb643fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:19.397803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf8a4e-742c-426e-b467-ce12bb643fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:19.398089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf8a4e-742c-426e-b467-ce12bb643fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:19.398875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf8a4e-742c-426e-b467-ce12bb643fc3 admin admin] PATH_INFO: `/v3/projects/64ba5f531af540d9940802e61aca4f6e/users/341af8d25c084be5aa82f23f33933d92/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:19.401057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf8a4e-742c-426e-b467-ce12bb643fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ba5f531af540d9940802e61aca4f6e, user_id=341af8d25c084be5aa82f23f33933d92, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:19.403792 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf8a4e-742c-426e-b467-ce12bb643fc3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:19.415149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1017/2054] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:19 2022] PUT /identity/v3/projects/64ba5f531af540d9940802e61aca4f6e/users/341af8d25c084be5aa82f23f33933d92/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:19.426258 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974252f-5c2c-4dce-a9f6-cb47f39f765e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:19.426566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974252f-5c2c-4dce-a9f6-cb47f39f765e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:19.426838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974252f-5c2c-4dce-a9f6-cb47f39f765e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:19.441973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:19.457613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2974252f-5c2c-4dce-a9f6-cb47f39f765e None None] MFA Rules not processed for user `341af8d25c084be5aa82f23f33933d92`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:19.458850 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2974252f-5c2c-4dce-a9f6-cb47f39f765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:19.506148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:19.542015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1038/2055] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:19.554265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf8a4e-742c-426e-b467-ce12bb643fc3 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:19.556045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-35bf8a4e-742c-426e-b467-ce12bb643fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:19.616231 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf8a4e-742c-426e-b467-ce12bb643fc3 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:19.620113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e2a57-6ffd-4339-bbcc-98373621b957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:19.620727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e2a57-6ffd-4339-bbcc-98373621b957 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:19.621036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e2a57-6ffd-4339-bbcc-98373621b957 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:19.621389 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e2a57-6ffd-4339-bbcc-98373621b957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:19.643465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:19.681355 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e2a57-6ffd-4339-bbcc-98373621b957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:19.682829 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e2a57-6ffd-4339-bbcc-98373621b957 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:19.692039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1018/2056] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:29.552882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=31 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:29.701638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=25 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:30.525566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974252f-5c2c-4dce-a9f6-cb47f39f765e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:30.527251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2974252f-5c2c-4dce-a9f6-cb47f39f765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:30.600938 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974252f-5c2c-4dce-a9f6-cb47f39f765e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:30.605767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:30.606427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:30.606731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:30.606995 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] PATH_INFO: `/v3/users/7e91fbe205fc4f0a89ba597c29081a7d` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:30.608251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e91fbe205fc4f0a89ba597c29081a7d)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:30.610268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:30.826039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e91fbe205fc4f0a89ba597c29081a7d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:30.826720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e91fbe205fc4f0a89ba597c29081a7d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:30.833746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e91fbe205fc4f0a89ba597c29081a7d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:30.906421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e91fbe205fc4f0a89ba597c29081a7d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:30.907118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e91fbe205fc4f0a89ba597c29081a7d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:30.940752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aeec2-5777-40c0-984b-c21661da9bff None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:30.941192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aeec2-5777-40c0-984b-c21661da9bff None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:30.945209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aeec2-5777-40c0-984b-c21661da9bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:30.945585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1039/2057] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:30 2022] DELETE /identity/v3/users/7e91fbe205fc4f0a89ba597c29081a7d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:30.975919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-751aeec2-5777-40c0-984b-c21661da9bff None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:30.977179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-751aeec2-5777-40c0-984b-c21661da9bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:31.043641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1019/2058] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:31.053435 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:31.054967 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:31.114838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c6479-1d05-4ddf-854a-46f4088b6867 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:31.119069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bd938-b576-4a17-9830-003aea330f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:31.119822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bd938-b576-4a17-9830-003aea330f58 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:31.120107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bd938-b576-4a17-9830-003aea330f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:31.120378 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bd938-b576-4a17-9830-003aea330f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:31.129711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042bd938-b576-4a17-9830-003aea330f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:31.131132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042bd938-b576-4a17-9830-003aea330f58 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:31.140574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1040/2059] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:31.311318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751aeec2-5777-40c0-984b-c21661da9bff None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:31.312780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-751aeec2-5777-40c0-984b-c21661da9bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:31.334377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751aeec2-5777-40c0-984b-c21661da9bff None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:31.342696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b0dc6-c888-46b5-9e07-7cd68e9f557e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:31.343627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b0dc6-c888-46b5-9e07-7cd68e9f557e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:31.344074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b0dc6-c888-46b5-9e07-7cd68e9f557e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:31.344642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b0dc6-c888-46b5-9e07-7cd68e9f557e admin admin] PATH_INFO: `/v3/users/6b8c2c4bdd614aa9b9f8cb286ed5ad55` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:31.345987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b0dc6-c888-46b5-9e07-7cd68e9f557e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b8c2c4bdd614aa9b9f8cb286ed5ad55)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:31.348344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b0dc6-c888-46b5-9e07-7cd68e9f557e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:31.587543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bd938-b576-4a17-9830-003aea330f58 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:31.589147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-042bd938-b576-4a17-9830-003aea330f58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:31.598620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-cf7b0dc6-c888-46b5-9e07-7cd68e9f557e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b8c2c4bdd614aa9b9f8cb286ed5ad55'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:31.603263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bd938-b576-4a17-9830-003aea330f58 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:31.607810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-cf7b0dc6-c888-46b5-9e07-7cd68e9f557e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b8c2c4bdd614aa9b9f8cb286ed5ad55'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:31.609261 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:31.610107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:31.610514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:31.610862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] PATH_INFO: `/v3/projects/8155dc06826e4156b281dc6f0e6ccda6` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:31.612124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8155dc06826e4156b281dc6f0e6ccda6)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:31.617707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:31.631726 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:31.688821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-cf7b0dc6-c888-46b5-9e07-7cd68e9f557e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b8c2c4bdd614aa9b9f8cb286ed5ad55'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:31.689515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-cf7b0dc6-c888-46b5-9e07-7cd68e9f557e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b8c2c4bdd614aa9b9f8cb286ed5ad55'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:31.704669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-cf7b0dc6-c888-46b5-9e07-7cd68e9f557e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b8c2c4bdd614aa9b9f8cb286ed5ad55'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:31.712762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1020/2060] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:31 2022] DELETE /identity/v3/users/6b8c2c4bdd614aa9b9f8cb286ed5ad55 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:31.717569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab99de-14e0-4270-b317-1cf08451081e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:31.717862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab99de-14e0-4270-b317-1cf08451081e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:31.718140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab99de-14e0-4270-b317-1cf08451081e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:31.737898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-24ab99de-14e0-4270-b317-1cf08451081e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:31.739166 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-24ab99de-14e0-4270-b317-1cf08451081e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:31.775647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8155dc06826e4156b281dc6f0e6ccda6', 'request_id': 'req-e973bf45-9c97-4c37-9afa-04dcceda8bc9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:31.783863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8155dc06826e4156b281dc6f0e6ccda6', 'request_id': 'req-e973bf45-9c97-4c37-9afa-04dcceda8bc9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:31.792244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8155dc06826e4156b281dc6f0e6ccda6', 'request_id': 'req-e973bf45-9c97-4c37-9afa-04dcceda8bc9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:31.807826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1021/2061] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:31.817473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab99de-14e0-4270-b317-1cf08451081e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:31.818856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-24ab99de-14e0-4270-b317-1cf08451081e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:31.833621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:31.879668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab99de-14e0-4270-b317-1cf08451081e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:31.901064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] The token cache is being invalidate because project 8155dc06826e4156b281dc6f0e6ccda6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:31.901757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:31.902893 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:31.904710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1041/2062] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:31 2022] DELETE /identity/v3/projects/8155dc06826e4156b281dc6f0e6ccda6 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:31.952855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895176d-23df-4f9c-99b6-43e2df04d585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:31.953590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895176d-23df-4f9c-99b6-43e2df04d585 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:31.954060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895176d-23df-4f9c-99b6-43e2df04d585 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:31.954347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895176d-23df-4f9c-99b6-43e2df04d585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:31.956301 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7895176d-23df-4f9c-99b6-43e2df04d585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:32.059508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895176d-23df-4f9c-99b6-43e2df04d585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:32.061888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895176d-23df-4f9c-99b6-43e2df04d585 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:32.074872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1022/2063] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:14:32.726357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:32.727956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:32.741871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973bf45-9c97-4c37-9afa-04dcceda8bc9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:32.751331 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:32.752035 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:32.754794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:32.754794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] PATH_INFO: `/v3/projects/b1a81d39608a400a8ba5c808c1559700` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:32.754794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1a81d39608a400a8ba5c808c1559700)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:32.755837 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:32.892509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1a81d39608a400a8ba5c808c1559700', 'request_id': 'req-198281b4-983b-416a-bc6e-b3a31fc7ce8a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:32.898064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1a81d39608a400a8ba5c808c1559700', 'request_id': 'req-198281b4-983b-416a-bc6e-b3a31fc7ce8a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:32.909849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1a81d39608a400a8ba5c808c1559700', 'request_id': 'req-198281b4-983b-416a-bc6e-b3a31fc7ce8a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:33.001706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] The token cache is being invalidate because project b1a81d39608a400a8ba5c808c1559700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:33.001706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:33.001706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:33.001706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1042/2064] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:32 2022] DELETE /identity/v3/projects/b1a81d39608a400a8ba5c808c1559700 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:33.004701 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895176d-23df-4f9c-99b6-43e2df04d585 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:33.015401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895176d-23df-4f9c-99b6-43e2df04d585 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:33.084805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27278930-3529-4a65-8995-b2752a4aa462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:33.085515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278930-3529-4a65-8995-b2752a4aa462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:33.085795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278930-3529-4a65-8995-b2752a4aa462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:33.086074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278930-3529-4a65-8995-b2752a4aa462 admin admin] PATH_INFO: `/v3/users/341af8d25c084be5aa82f23f33933d92` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:33.087291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27278930-3529-4a65-8995-b2752a4aa462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=341af8d25c084be5aa82f23f33933d92)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:33.089612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27278930-3529-4a65-8995-b2752a4aa462 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:33.113471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:33.297692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-27278930-3529-4a65-8995-b2752a4aa462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '341af8d25c084be5aa82f23f33933d92'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:33.304469 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-27278930-3529-4a65-8995-b2752a4aa462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '341af8d25c084be5aa82f23f33933d92'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:33.392624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-27278930-3529-4a65-8995-b2752a4aa462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '341af8d25c084be5aa82f23f33933d92'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:33.393344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-27278930-3529-4a65-8995-b2752a4aa462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '341af8d25c084be5aa82f23f33933d92'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:33.407136 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-27278930-3529-4a65-8995-b2752a4aa462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '341af8d25c084be5aa82f23f33933d92'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:33.416785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1023/2065] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:33 2022] DELETE /identity/v3/users/341af8d25c084be5aa82f23f33933d92 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:33.934523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:33.939594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:33.954048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198281b4-983b-416a-bc6e-b3a31fc7ce8a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:34.020640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:34.021332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.021620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.021899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] PATH_INFO: `/v3/projects/64ba5f531af540d9940802e61aca4f6e` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.023079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64ba5f531af540d9940802e61aca4f6e)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:34.025007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:34.117806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:34.137489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64ba5f531af540d9940802e61aca4f6e', 'request_id': 'req-d6360b2e-5739-4d48-94ca-86165852ce41'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.146059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64ba5f531af540d9940802e61aca4f6e', 'request_id': 'req-d6360b2e-5739-4d48-94ca-86165852ce41'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.156691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64ba5f531af540d9940802e61aca4f6e', 'request_id': 'req-d6360b2e-5739-4d48-94ca-86165852ce41'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.214931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] The token cache is being invalidate because project 64ba5f531af540d9940802e61aca4f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:34.215229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.215881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.217707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1043/2066] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:33 2022] DELETE /identity/v3/projects/64ba5f531af540d9940802e61aca4f6e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:34.222648 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136adb78-85d4-4e7d-8ce2-709413658ce9 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.223167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136adb78-85d4-4e7d-8ce2-709413658ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.223167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136adb78-85d4-4e7d-8ce2-709413658ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.243851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-136adb78-85d4-4e7d-8ce2-709413658ce9 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:34.245733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-136adb78-85d4-4e7d-8ce2-709413658ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:34.282089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:34.283587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:34.309776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1024/2067] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:34.317677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0a06a-2acb-4ce6-b2d3-72538318b10d None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.317967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0a06a-2acb-4ce6-b2d3-72538318b10d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.318241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0a06a-2acb-4ce6-b2d3-72538318b10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.337111 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-cbb0a06a-2acb-4ce6-b2d3-72538318b10d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:34.338873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cbb0a06a-2acb-4ce6-b2d3-72538318b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:34.347332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1025/2068] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:34.349658 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6360b2e-5739-4d48-94ca-86165852ce41 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:34.351682 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0a06a-2acb-4ce6-b2d3-72538318b10d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:34.355266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cecade-d948-4924-9da3-eafaf74a4dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:34.355958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cecade-d948-4924-9da3-eafaf74a4dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.356210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cecade-d948-4924-9da3-eafaf74a4dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.356515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cecade-d948-4924-9da3-eafaf74a4dfe admin admin] PATH_INFO: `/v3/users/168247d50fd04c08907841280bb8ec6b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.358047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cecade-d948-4924-9da3-eafaf74a4dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=168247d50fd04c08907841280bb8ec6b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:34.360043 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cecade-d948-4924-9da3-eafaf74a4dfe admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:34.361977 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0a06a-2acb-4ce6-b2d3-72538318b10d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:34.367397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700acfc-4b57-48f1-aed3-9c2c23b7fdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:34.368004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700acfc-4b57-48f1-aed3-9c2c23b7fdbe admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.368295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700acfc-4b57-48f1-aed3-9c2c23b7fdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.368578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700acfc-4b57-48f1-aed3-9c2c23b7fdbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.369345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700acfc-4b57-48f1-aed3-9c2c23b7fdbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:34.369656 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700acfc-4b57-48f1-aed3-9c2c23b7fdbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:34.371267 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700acfc-4b57-48f1-aed3-9c2c23b7fdbe admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:34.383396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1026/2069] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:14:34.387670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700acfc-4b57-48f1-aed3-9c2c23b7fdbe admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:34.397342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700acfc-4b57-48f1-aed3-9c2c23b7fdbe admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:34.402566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfe1d6-16c1-4af7-9c1f-8914e2321ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:34.403159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfe1d6-16c1-4af7-9c1f-8914e2321ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.403445 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfe1d6-16c1-4af7-9c1f-8914e2321ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.403723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfe1d6-16c1-4af7-9c1f-8914e2321ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.404389 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfe1d6-16c1-4af7-9c1f-8914e2321ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:34.406305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfe1d6-16c1-4af7-9c1f-8914e2321ea1 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:34.501352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1027/2070] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:34 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:34.502608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9e1b6-c609-4174-b562-d0c0d4450f45 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.502913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9e1b6-c609-4174-b562-d0c0d4450f45 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.503323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9e1b6-c609-4174-b562-d0c0d4450f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.521981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-55f9e1b6-c609-4174-b562-d0c0d4450f45 None None] MFA Rules not processed for user `a794c5d63c0f499db264113d64ae63b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:34.523299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-55f9e1b6-c609-4174-b562-d0c0d4450f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:34.536337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e5cecade-d948-4924-9da3-eafaf74a4dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '168247d50fd04c08907841280bb8ec6b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.537049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e5cecade-d948-4924-9da3-eafaf74a4dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '168247d50fd04c08907841280bb8ec6b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.544679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e5cecade-d948-4924-9da3-eafaf74a4dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '168247d50fd04c08907841280bb8ec6b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.578752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1028/2071] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:34.579682 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9e1b6-c609-4174-b562-d0c0d4450f45 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:34.583418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:34.604546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e5cecade-d948-4924-9da3-eafaf74a4dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '168247d50fd04c08907841280bb8ec6b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.605364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e5cecade-d948-4924-9da3-eafaf74a4dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '168247d50fd04c08907841280bb8ec6b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:34.637827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1044/2072] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:34 2022] DELETE /identity/v3/users/168247d50fd04c08907841280bb8ec6b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:34.642129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f990fa9-c40d-4930-a7f1-11ebaf8ca593 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.642665 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f990fa9-c40d-4930-a7f1-11ebaf8ca593 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.642928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f990fa9-c40d-4930-a7f1-11ebaf8ca593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.657807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9e1b6-c609-4174-b562-d0c0d4450f45 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:34.663356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2f990fa9-c40d-4930-a7f1-11ebaf8ca593 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:34.664629 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2f990fa9-c40d-4930-a7f1-11ebaf8ca593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:34.712155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:34.732893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef3fd8-19fe-440a-97dd-b394d8fcb7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:34.733734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef3fd8-19fe-440a-97dd-b394d8fcb7bf admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.734125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef3fd8-19fe-440a-97dd-b394d8fcb7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.737349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef3fd8-19fe-440a-97dd-b394d8fcb7bf admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.737349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef3fd8-19fe-440a-97dd-b394d8fcb7bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:34.737349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef3fd8-19fe-440a-97dd-b394d8fcb7bf admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:34.756873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1045/2073] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:34.774216 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f990fa9-c40d-4930-a7f1-11ebaf8ca593 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:34.776204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2f990fa9-c40d-4930-a7f1-11ebaf8ca593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:34.836496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f990fa9-c40d-4930-a7f1-11ebaf8ca593 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:34.840881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21422a-e6b0-4312-aec3-30d5169dc817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:34.841817 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21422a-e6b0-4312-aec3-30d5169dc817 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.842133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21422a-e6b0-4312-aec3-30d5169dc817 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.848104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21422a-e6b0-4312-aec3-30d5169dc817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.851930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21422a-e6b0-4312-aec3-30d5169dc817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:34.853551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21422a-e6b0-4312-aec3-30d5169dc817 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:34.863196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1046/2074] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:34.873398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1029/2075] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:34 2022] POST /identity/v3/users => generated 288 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:34.877355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21422a-e6b0-4312-aec3-30d5169dc817 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:34.878872 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1a21422a-e6b0-4312-aec3-30d5169dc817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:34.892347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21422a-e6b0-4312-aec3-30d5169dc817 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:34.898263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55928c16-df50-4dad-a4cd-54e9c3296a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:34.898889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55928c16-df50-4dad-a4cd-54e9c3296a7f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.899173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55928c16-df50-4dad-a4cd-54e9c3296a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.899449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55928c16-df50-4dad-a4cd-54e9c3296a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.900235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55928c16-df50-4dad-a4cd-54e9c3296a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:34.902490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55928c16-df50-4dad-a4cd-54e9c3296a7f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:34.914132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1047/2076] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:34.917677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef3fd8-19fe-440a-97dd-b394d8fcb7bf admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:34.928213 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef3fd8-19fe-440a-97dd-b394d8fcb7bf admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:34.932929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfc5ad-ce1e-454d-8daa-4a99f5440c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:34.933627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc5ad-ce1e-454d-8daa-4a99f5440c92 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.933914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc5ad-ce1e-454d-8daa-4a99f5440c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.934195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc5ad-ce1e-454d-8daa-4a99f5440c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.934915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfc5ad-ce1e-454d-8daa-4a99f5440c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:34.936547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfc5ad-ce1e-454d-8daa-4a99f5440c92 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:34.949002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1030/2077] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:34.951795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55928c16-df50-4dad-a4cd-54e9c3296a7f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:34.964099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55928c16-df50-4dad-a4cd-54e9c3296a7f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:34.969812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919adb7c-1c87-421d-a964-a88b7bea652b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:34.970782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919adb7c-1c87-421d-a964-a88b7bea652b admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:34.971496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919adb7c-1c87-421d-a964-a88b7bea652b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:34.971974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919adb7c-1c87-421d-a964-a88b7bea652b admin admin] PATH_INFO: `/v3/projects/e048b2ccc0554f30b5180dc0955f0ff3/users/e63e9781272d4344ba4248f444e19d87/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:34.990847 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919adb7c-1c87-421d-a964-a88b7bea652b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e048b2ccc0554f30b5180dc0955f0ff3, user_id=e63e9781272d4344ba4248f444e19d87, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:34.993575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919adb7c-1c87-421d-a964-a88b7bea652b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:35.064425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1048/2078] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:34 2022] PUT /identity/v3/projects/e048b2ccc0554f30b5180dc0955f0ff3/users/e63e9781272d4344ba4248f444e19d87/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:35.137788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66619c6f-a306-4c50-8daa-3af4f31b2de6 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.138076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66619c6f-a306-4c50-8daa-3af4f31b2de6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.138356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66619c6f-a306-4c50-8daa-3af4f31b2de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.169966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-66619c6f-a306-4c50-8daa-3af4f31b2de6 None None] MFA Rules not processed for user `e63e9781272d4344ba4248f444e19d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:35.172235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-66619c6f-a306-4c50-8daa-3af4f31b2de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:35.273070 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1031/2079] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:35.283416 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919adb7c-1c87-421d-a964-a88b7bea652b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:35.286047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-919adb7c-1c87-421d-a964-a88b7bea652b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:35.360060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919adb7c-1c87-421d-a964-a88b7bea652b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:35.365227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22949962-c7e0-49fd-8a8c-070ba31ead22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:35.366021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22949962-c7e0-49fd-8a8c-070ba31ead22 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.366592 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22949962-c7e0-49fd-8a8c-070ba31ead22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.366943 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22949962-c7e0-49fd-8a8c-070ba31ead22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.383944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22949962-c7e0-49fd-8a8c-070ba31ead22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:35.386610 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22949962-c7e0-49fd-8a8c-070ba31ead22 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:35.396942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1049/2080] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:35.646875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66619c6f-a306-4c50-8daa-3af4f31b2de6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:35.648362 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-66619c6f-a306-4c50-8daa-3af4f31b2de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:35.686983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:35.736591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66619c6f-a306-4c50-8daa-3af4f31b2de6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:35.747099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:35.747772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:35.748066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:35.748401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] PATH_INFO: `/v3/projects/ba7ca9875a34453cb582aba7770ba4b6` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:35.749574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba7ca9875a34453cb582aba7770ba4b6)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:35.752097 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:35.938926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba7ca9875a34453cb582aba7770ba4b6', 'request_id': 'req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:35.944984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba7ca9875a34453cb582aba7770ba4b6', 'request_id': 'req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:36.020840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba7ca9875a34453cb582aba7770ba4b6', 'request_id': 'req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:36.035356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] The token cache is being invalidate because project ba7ca9875a34453cb582aba7770ba4b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:36.035655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:36.036346 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:36.047186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1032/2081] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:35 2022] DELETE /identity/v3/projects/ba7ca9875a34453cb582aba7770ba4b6 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:36.051400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22949962-c7e0-49fd-8a8c-070ba31ead22 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:36.063385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22949962-c7e0-49fd-8a8c-070ba31ead22 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:36.125359 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:36.142484 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf1901-c083-40cc-8b20-77738ce395d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:36.143271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf1901-c083-40cc-8b20-77738ce395d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:36.143623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf1901-c083-40cc-8b20-77738ce395d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:36.143986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf1901-c083-40cc-8b20-77738ce395d4 admin admin] PATH_INFO: `/v3/users/9bf87dc8311e4596b6203fd37294ace9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:36.145527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf1901-c083-40cc-8b20-77738ce395d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf87dc8311e4596b6203fd37294ace9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:36.147515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf1901-c083-40cc-8b20-77738ce395d4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:36.325650 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-25cf1901-c083-40cc-8b20-77738ce395d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf87dc8311e4596b6203fd37294ace9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:36.326513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-25cf1901-c083-40cc-8b20-77738ce395d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf87dc8311e4596b6203fd37294ace9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:36.333987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-25cf1901-c083-40cc-8b20-77738ce395d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf87dc8311e4596b6203fd37294ace9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:36.456052 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-25cf1901-c083-40cc-8b20-77738ce395d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf87dc8311e4596b6203fd37294ace9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:36.456771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-25cf1901-c083-40cc-8b20-77738ce395d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf87dc8311e4596b6203fd37294ace9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:36.476021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1050/2082] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:36 2022] DELETE /identity/v3/users/9bf87dc8311e4596b6203fd37294ace9 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:37.106406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:37.106406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:37.125508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ed3a3-d54a-475e-9fd5-2dde0a7967e1 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:37.207669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:37.208386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:37.208907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:37.209221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] PATH_INFO: `/v3/projects/02b2b143dff740fea29abd444e3a0d52` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:37.210780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02b2b143dff740fea29abd444e3a0d52)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:37.212475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:37.339776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02b2b143dff740fea29abd444e3a0d52', 'request_id': 'req-53166d34-7b8e-43a9-9331-4b6e0cda4977'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:37.341367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:37.347975 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02b2b143dff740fea29abd444e3a0d52', 'request_id': 'req-53166d34-7b8e-43a9-9331-4b6e0cda4977'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:37.424881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02b2b143dff740fea29abd444e3a0d52', 'request_id': 'req-53166d34-7b8e-43a9-9331-4b6e0cda4977'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:37.437727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] The token cache is being invalidate because project 02b2b143dff740fea29abd444e3a0d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:37.438162 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:37.439034 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-53166d34-7b8e-43a9-9331-4b6e0cda4977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:37.442698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1033/2083] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:37 2022] DELETE /identity/v3/projects/02b2b143dff740fea29abd444e3a0d52 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:37.448568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cdb75-4ead-41e6-b0fd-f056148a82d6 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:37.449137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cdb75-4ead-41e6-b0fd-f056148a82d6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:37.449467 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cdb75-4ead-41e6-b0fd-f056148a82d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:37.471848 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-dd7cdb75-4ead-41e6-b0fd-f056148a82d6 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:37.473323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dd7cdb75-4ead-41e6-b0fd-f056148a82d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:37.557625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1051/2084] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:37.565630 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4146b1-6865-4f27-bdcc-31408be22479 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:37.565918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4146b1-6865-4f27-bdcc-31408be22479 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:37.566195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4146b1-6865-4f27-bdcc-31408be22479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:37.587434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9c4146b1-6865-4f27-bdcc-31408be22479 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:37.590505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9c4146b1-6865-4f27-bdcc-31408be22479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:37.598172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1034/2085] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:37.602461 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cdb75-4ead-41e6-b0fd-f056148a82d6 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:37.613975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cdb75-4ead-41e6-b0fd-f056148a82d6 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:37.618917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcb4d5-820d-4807-b8a9-d425724daa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:37.620150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcb4d5-820d-4807-b8a9-d425724daa75 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:37.620433 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcb4d5-820d-4807-b8a9-d425724daa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:37.620709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcb4d5-820d-4807-b8a9-d425724daa75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:37.621523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcb4d5-820d-4807-b8a9-d425724daa75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:37.621875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcb4d5-820d-4807-b8a9-d425724daa75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:37.623573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcb4d5-820d-4807-b8a9-d425724daa75 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:37.636940 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1052/2086] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:37.645066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4146b1-6865-4f27-bdcc-31408be22479 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:37.655786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4146b1-6865-4f27-bdcc-31408be22479 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:37.661063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df5c9c-2aed-4eca-bdea-3b3ea4eb3f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:37.665289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df5c9c-2aed-4eca-bdea-3b3ea4eb3f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:37.665289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df5c9c-2aed-4eca-bdea-3b3ea4eb3f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:37.665289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df5c9c-2aed-4eca-bdea-3b3ea4eb3f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:37.665289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df5c9c-2aed-4eca-bdea-3b3ea4eb3f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:37.665289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df5c9c-2aed-4eca-bdea-3b3ea4eb3f55 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:37.832043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1035/2087] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:37 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:37.834972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcb4d5-820d-4807-b8a9-d425724daa75 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:37.853843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcb4d5-820d-4807-b8a9-d425724daa75 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:37.922762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:37.936202 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896f0a0-f566-489e-871e-aa11c14c834d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:37.936913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896f0a0-f566-489e-871e-aa11c14c834d admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:37.942511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896f0a0-f566-489e-871e-aa11c14c834d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:37.942511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896f0a0-f566-489e-871e-aa11c14c834d admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:37.942511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896f0a0-f566-489e-871e-aa11c14c834d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:37.942511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896f0a0-f566-489e-871e-aa11c14c834d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:38.034370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1053/2088] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:37 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:38.040020 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df5c9c-2aed-4eca-bdea-3b3ea4eb3f55 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:38.059576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df5c9c-2aed-4eca-bdea-3b3ea4eb3f55 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:38.066186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a2fcb-7d5d-47fc-b466-c9c98afce497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:38.066926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a2fcb-7d5d-47fc-b466-c9c98afce497 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:38.067357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a2fcb-7d5d-47fc-b466-c9c98afce497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:38.067709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a2fcb-7d5d-47fc-b466-c9c98afce497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:38.068553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a2fcb-7d5d-47fc-b466-c9c98afce497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:38.070314 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a2fcb-7d5d-47fc-b466-c9c98afce497 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:38.096375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1036/2089] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:38.100987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896f0a0-f566-489e-871e-aa11c14c834d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:38.117235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896f0a0-f566-489e-871e-aa11c14c834d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:38.124473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237fee2-9086-4413-9e5d-c019dfa0c2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:38.125382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237fee2-9086-4413-9e5d-c019dfa0c2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:38.125743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237fee2-9086-4413-9e5d-c019dfa0c2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:38.126088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237fee2-9086-4413-9e5d-c019dfa0c2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:38.126884 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237fee2-9086-4413-9e5d-c019dfa0c2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:38.129399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237fee2-9086-4413-9e5d-c019dfa0c2e4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:38.141262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1054/2090] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:38.145538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a2fcb-7d5d-47fc-b466-c9c98afce497 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:38.156885 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a2fcb-7d5d-47fc-b466-c9c98afce497 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:38.161306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e58cd-a7cd-4c91-9780-33790b5c81fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:38.162045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e58cd-a7cd-4c91-9780-33790b5c81fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:38.167386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e58cd-a7cd-4c91-9780-33790b5c81fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:38.167386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e58cd-a7cd-4c91-9780-33790b5c81fa admin admin] PATH_INFO: `/v3/projects/fa94416b76ed4a8dbb7e6a15d8cf6e3a/users/5deabdca4b8943b1843a2203747a0024/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:38.177904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e58cd-a7cd-4c91-9780-33790b5c81fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa94416b76ed4a8dbb7e6a15d8cf6e3a, user_id=5deabdca4b8943b1843a2203747a0024, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:38.180609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e58cd-a7cd-4c91-9780-33790b5c81fa admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:38.253918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1037/2091] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:38 2022] PUT /identity/v3/projects/fa94416b76ed4a8dbb7e6a15d8cf6e3a/users/5deabdca4b8943b1843a2203747a0024/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:38.269172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a71ce-1d52-4317-a5c0-c8c5e85d4ec3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:38.269842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a71ce-1d52-4317-a5c0-c8c5e85d4ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:38.270371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a71ce-1d52-4317-a5c0-c8c5e85d4ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:38.307484 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-912a71ce-1d52-4317-a5c0-c8c5e85d4ec3 None None] MFA Rules not processed for user `5deabdca4b8943b1843a2203747a0024`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:38.308876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-912a71ce-1d52-4317-a5c0-c8c5e85d4ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:38.445024 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1055/2092] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:38.447192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e58cd-a7cd-4c91-9780-33790b5c81fa admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:38.518750 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e58cd-a7cd-4c91-9780-33790b5c81fa admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:38.523939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8cd9a-ed9c-4cac-a8b1-5c4a44a11715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:38.524685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8cd9a-ed9c-4cac-a8b1-5c4a44a11715 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:38.525147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8cd9a-ed9c-4cac-a8b1-5c4a44a11715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:38.525807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8cd9a-ed9c-4cac-a8b1-5c4a44a11715 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:38.526298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8cd9a-ed9c-4cac-a8b1-5c4a44a11715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:38.528356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8cd9a-ed9c-4cac-a8b1-5c4a44a11715 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:38.661535 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1038/2093] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:38 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:38.666344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a71ce-1d52-4317-a5c0-c8c5e85d4ec3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:38.678061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a71ce-1d52-4317-a5c0-c8c5e85d4ec3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:38.770803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4d2f4-03ca-4ba4-925e-4bd0ad745f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:38.772169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4d2f4-03ca-4ba4-925e-4bd0ad745f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:38.772169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4d2f4-03ca-4ba4-925e-4bd0ad745f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:38.772169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4d2f4-03ca-4ba4-925e-4bd0ad745f38 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:38.772811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4d2f4-03ca-4ba4-925e-4bd0ad745f38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:38.774811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4d2f4-03ca-4ba4-925e-4bd0ad745f38 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:38.904990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1056/2094] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:38 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:38.907692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8cd9a-ed9c-4cac-a8b1-5c4a44a11715 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:38.933227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8cd9a-ed9c-4cac-a8b1-5c4a44a11715 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:38.938379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d349-8be9-4c29-9cc2-e7271fbf0b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:38.939117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d349-8be9-4c29-9cc2-e7271fbf0b97 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:38.939408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d349-8be9-4c29-9cc2-e7271fbf0b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:38.939793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d349-8be9-4c29-9cc2-e7271fbf0b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:38.940408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801d349-8be9-4c29-9cc2-e7271fbf0b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:38.942639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801d349-8be9-4c29-9cc2-e7271fbf0b97 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:38.957092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1039/2095] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:38.961205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4d2f4-03ca-4ba4-925e-4bd0ad745f38 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:38.985970 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4d2f4-03ca-4ba4-925e-4bd0ad745f38 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:38.997196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca58862-c861-433c-9882-d299319773b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:39.000465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca58862-c861-433c-9882-d299319773b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:39.000465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca58862-c861-433c-9882-d299319773b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:39.000465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca58862-c861-433c-9882-d299319773b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:39.000465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca58862-c861-433c-9882-d299319773b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:39.001713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca58862-c861-433c-9882-d299319773b2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:39.017016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1057/2096] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:39.018508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d349-8be9-4c29-9cc2-e7271fbf0b97 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:39.030238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d349-8be9-4c29-9cc2-e7271fbf0b97 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:39.035764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2347f-acde-4abc-a603-841640b6a698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:39.037040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2347f-acde-4abc-a603-841640b6a698 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:39.045832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2347f-acde-4abc-a603-841640b6a698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:39.045832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2347f-acde-4abc-a603-841640b6a698 admin admin] PATH_INFO: `/v3/projects/118fdddd109e4a94933c1bbf58d2586a/users/629137049b994cd5adf332a4da25a18d/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:39.061534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2347f-acde-4abc-a603-841640b6a698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118fdddd109e4a94933c1bbf58d2586a, user_id=629137049b994cd5adf332a4da25a18d, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:39.066481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2347f-acde-4abc-a603-841640b6a698 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:39.163714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1040/2097] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:39 2022] PUT /identity/v3/projects/118fdddd109e4a94933c1bbf58d2586a/users/629137049b994cd5adf332a4da25a18d/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:39.174854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c561-11a8-4072-8324-765538d33edc None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:39.177615 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c561-11a8-4072-8324-765538d33edc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:39.177615 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c561-11a8-4072-8324-765538d33edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:39.224327 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3fc6c561-11a8-4072-8324-765538d33edc None None] MFA Rules not processed for user `629137049b994cd5adf332a4da25a18d`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:39.229077 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3fc6c561-11a8-4072-8324-765538d33edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:39.298499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:39.379640 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2347f-acde-4abc-a603-841640b6a698 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:39.391941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1058/2098] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:39.434871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:39.484003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2347f-acde-4abc-a603-841640b6a698 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:39.492728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75382234-2782-401c-a084-ccef704a899e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:39.493447 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75382234-2782-401c-a084-ccef704a899e admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:39.495638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75382234-2782-401c-a084-ccef704a899e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:39.495988 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75382234-2782-401c-a084-ccef704a899e admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:39.496706 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75382234-2782-401c-a084-ccef704a899e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:39.498932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75382234-2782-401c-a084-ccef704a899e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:39.591610 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1041/2099] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:39 2022] POST /identity/v3/projects => generated 387 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:39.595881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6c561-11a8-4072-8324-765538d33edc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:39.609796 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6c561-11a8-4072-8324-765538d33edc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:39.708150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc6d68-b3d2-4f82-8379-9b1952b725ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:39.708986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc6d68-b3d2-4f82-8379-9b1952b725ca admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:39.709569 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc6d68-b3d2-4f82-8379-9b1952b725ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:39.710007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc6d68-b3d2-4f82-8379-9b1952b725ca admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:39.710855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc6d68-b3d2-4f82-8379-9b1952b725ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:39.713537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc6d68-b3d2-4f82-8379-9b1952b725ca admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:39.825516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1059/2100] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:39 2022] POST /identity/v3/users => generated 300 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:39.829111 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75382234-2782-401c-a084-ccef704a899e admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:39.841713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75382234-2782-401c-a084-ccef704a899e admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:39.846666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9357593-0575-4911-a814-d7dd57687bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:39.847384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9357593-0575-4911-a814-d7dd57687bbe admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:39.847789 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9357593-0575-4911-a814-d7dd57687bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:39.848070 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9357593-0575-4911-a814-d7dd57687bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:39.848782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9357593-0575-4911-a814-d7dd57687bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:39.850455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9357593-0575-4911-a814-d7dd57687bbe admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:39.865012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1042/2101] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:14:39.870125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc6d68-b3d2-4f82-8379-9b1952b725ca admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:39.892952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc6d68-b3d2-4f82-8379-9b1952b725ca admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:39.916494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c7dbd-9a21-4ed1-91dd-142a75e1fc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:39.920801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c7dbd-9a21-4ed1-91dd-142a75e1fc0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:39.920801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c7dbd-9a21-4ed1-91dd-142a75e1fc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:39.920801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c7dbd-9a21-4ed1-91dd-142a75e1fc0d admin admin] PATH_INFO: `/v3/projects/f0d204a0ff1643f4b423dd4a9a1881b2/users/0886e639d68343e4b3fd506e2d98723e/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:39.944476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c7dbd-9a21-4ed1-91dd-142a75e1fc0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d204a0ff1643f4b423dd4a9a1881b2, user_id=0886e639d68343e4b3fd506e2d98723e, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:39.948431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c7dbd-9a21-4ed1-91dd-142a75e1fc0d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:40.079563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1060/2102] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:39 2022] PUT /identity/v3/projects/f0d204a0ff1643f4b423dd4a9a1881b2/users/0886e639d68343e4b3fd506e2d98723e/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:40.083864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9357593-0575-4911-a814-d7dd57687bbe admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:40.112481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9357593-0575-4911-a814-d7dd57687bbe admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:40.218996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873c26d-abae-441c-9929-9f94bf7f29a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:40.218996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873c26d-abae-441c-9929-9f94bf7f29a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:40.218996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873c26d-abae-441c-9929-9f94bf7f29a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:40.219986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873c26d-abae-441c-9929-9f94bf7f29a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:40.219986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873c26d-abae-441c-9929-9f94bf7f29a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:40.223103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873c26d-abae-441c-9929-9f94bf7f29a4 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:40.236523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1043/2103] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:40.240608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c7dbd-9a21-4ed1-91dd-142a75e1fc0d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:40.252127 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c7dbd-9a21-4ed1-91dd-142a75e1fc0d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:40.257755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51411adf-c0eb-4f16-99f5-9f411660a4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:40.258436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51411adf-c0eb-4f16-99f5-9f411660a4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:40.258728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51411adf-c0eb-4f16-99f5-9f411660a4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:40.259107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51411adf-c0eb-4f16-99f5-9f411660a4f5 admin admin] PATH_INFO: `/v3/projects/f0d204a0ff1643f4b423dd4a9a1881b2/users/0886e639d68343e4b3fd506e2d98723e/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:40.261621 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51411adf-c0eb-4f16-99f5-9f411660a4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d204a0ff1643f4b423dd4a9a1881b2, user_id=0886e639d68343e4b3fd506e2d98723e, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:40.264294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51411adf-c0eb-4f16-99f5-9f411660a4f5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:40.276668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1061/2104] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:40 2022] PUT /identity/v3/projects/f0d204a0ff1643f4b423dd4a9a1881b2/users/0886e639d68343e4b3fd506e2d98723e/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:40.287949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8d8cd-0d78-4f1c-af65-60139e6f6e58 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:40.288518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8d8cd-0d78-4f1c-af65-60139e6f6e58 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:40.295186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8d8cd-0d78-4f1c-af65-60139e6f6e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:40.324547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-78c8d8cd-0d78-4f1c-af65-60139e6f6e58 None None] MFA Rules not processed for user `0886e639d68343e4b3fd506e2d98723e`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:40.325807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-78c8d8cd-0d78-4f1c-af65-60139e6f6e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:40.406467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1044/2105] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:40.430451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51411adf-c0eb-4f16-99f5-9f411660a4f5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:40.432903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-51411adf-c0eb-4f16-99f5-9f411660a4f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:40.500885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51411adf-c0eb-4f16-99f5-9f411660a4f5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:40.501801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6bdc7e-d0eb-4c93-ab5b-4d01b3f59235 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:40.502168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6bdc7e-d0eb-4c93-ab5b-4d01b3f59235 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:40.502580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6bdc7e-d0eb-4c93-ab5b-4d01b3f59235 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:40.504265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1045/2106] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 18:14:40.505240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d04ff-98a1-4688-98e4-df8b1eadf696 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:40.505940 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d04ff-98a1-4688-98e4-df8b1eadf696 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:40.506299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d04ff-98a1-4688-98e4-df8b1eadf696 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:40.506649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d04ff-98a1-4688-98e4-df8b1eadf696 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:40.517498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d04ff-98a1-4688-98e4-df8b1eadf696 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:40.518462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d04ff-98a1-4688-98e4-df8b1eadf696 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:40.528677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1062/2107] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:40.544302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05e7e-d9ab-479e-be16-b5956154e1cd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:40.545168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05e7e-d9ab-479e-be16-b5956154e1cd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:40.548223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05e7e-d9ab-479e-be16-b5956154e1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:40.574070 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-05d05e7e-d9ab-479e-be16-b5956154e1cd None None] MFA Rules not processed for user `b20564a83953433290aa3fabf884370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:40.575542 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-05d05e7e-d9ab-479e-be16-b5956154e1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:40.598081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:40.656614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1046/2108] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:14:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:40.672574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d04ff-98a1-4688-98e4-df8b1eadf696 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:40.674195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c04d04ff-98a1-4688-98e4-df8b1eadf696 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:40.685968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d05e7e-d9ab-479e-be16-b5956154e1cd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:40.687696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-05d05e7e-d9ab-479e-be16-b5956154e1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:40.706054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d05e7e-d9ab-479e-be16-b5956154e1cd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:40.712429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62742f0-1213-423f-a90e-f3f310e27154 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:40.713982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62742f0-1213-423f-a90e-f3f310e27154 service placement] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:40.714455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62742f0-1213-423f-a90e-f3f310e27154 service placement] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:40.714784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62742f0-1213-423f-a90e-f3f310e27154 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:40.715379 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:40.729973 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62742f0-1213-423f-a90e-f3f310e27154 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:40.731752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62742f0-1213-423f-a90e-f3f310e27154 service placement] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:40.754485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d04ff-98a1-4688-98e4-df8b1eadf696 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:40.757712 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1047/2109] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:40.760391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:40.761080 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:40.761366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:40.761644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] PATH_INFO: `/v3/users/d96e9a9e40c54b6a9a7f8560e4846be5` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:40.762919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96e9a9e40c54b6a9a7f8560e4846be5)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:40.764843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:41.009412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96e9a9e40c54b6a9a7f8560e4846be5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.010543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96e9a9e40c54b6a9a7f8560e4846be5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.018627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96e9a9e40c54b6a9a7f8560e4846be5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.141605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96e9a9e40c54b6a9a7f8560e4846be5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.142445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96e9a9e40c54b6a9a7f8560e4846be5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.162545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1063/2110] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:40 2022] DELETE /identity/v3/users/d96e9a9e40c54b6a9a7f8560e4846be5 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:41.190415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62742f0-1213-423f-a90e-f3f310e27154 service placement] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:41.191996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f62742f0-1213-423f-a90e-f3f310e27154 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:41.206356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62742f0-1213-423f-a90e-f3f310e27154 service placement] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:41.273545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f46b5-0a58-4a33-8f44-38c7d785c556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:41.274204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f46b5-0a58-4a33-8f44-38c7d785c556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:41.274491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f46b5-0a58-4a33-8f44-38c7d785c556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:41.274774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f46b5-0a58-4a33-8f44-38c7d785c556 admin admin] PATH_INFO: `/v3/users/e63e9781272d4344ba4248f444e19d87` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:41.275975 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f46b5-0a58-4a33-8f44-38c7d785c556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e63e9781272d4344ba4248f444e19d87)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:41.277960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f46b5-0a58-4a33-8f44-38c7d785c556 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:41.441845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:41.443041 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:41.450128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c34f46b5-0a58-4a33-8f44-38c7d785c556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e63e9781272d4344ba4248f444e19d87'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.464303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ba604-dee0-4926-9798-8ee2957cc604 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:41.475574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c34f46b5-0a58-4a33-8f44-38c7d785c556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e63e9781272d4344ba4248f444e19d87'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.476060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:41.476060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:41.476060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:41.476060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] PATH_INFO: `/v3/projects/8ef52881d75f407a942bf960230ea22e` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:41.476581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef52881d75f407a942bf960230ea22e)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:41.476581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:41.552539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c34f46b5-0a58-4a33-8f44-38c7d785c556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e63e9781272d4344ba4248f444e19d87'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.553190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c34f46b5-0a58-4a33-8f44-38c7d785c556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e63e9781272d4344ba4248f444e19d87'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.567333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c34f46b5-0a58-4a33-8f44-38c7d785c556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e63e9781272d4344ba4248f444e19d87'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.573778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1048/2111] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:41 2022] DELETE /identity/v3/users/e63e9781272d4344ba4248f444e19d87 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:41.578274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb34481-e4ed-4799-a512-d49271cc9381 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:41.578563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb34481-e4ed-4799-a512-d49271cc9381 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:41.578843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb34481-e4ed-4799-a512-d49271cc9381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:41.601343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9eb34481-e4ed-4799-a512-d49271cc9381 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:41.602711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9eb34481-e4ed-4799-a512-d49271cc9381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:41.627932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:41.662801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef52881d75f407a942bf960230ea22e', 'request_id': 'req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.668046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef52881d75f407a942bf960230ea22e', 'request_id': 'req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.677348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef52881d75f407a942bf960230ea22e', 'request_id': 'req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.689133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1049/2112] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:41.704464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb34481-e4ed-4799-a512-d49271cc9381 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:41.705905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9eb34481-e4ed-4799-a512-d49271cc9381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:41.746587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] The token cache is being invalidate because project 8ef52881d75f407a942bf960230ea22e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:41.748679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.748679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:41.753450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1064/2113] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:41 2022] DELETE /identity/v3/projects/8ef52881d75f407a942bf960230ea22e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:41.753450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:41.753450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:41.784907 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:41.786757 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb34481-e4ed-4799-a512-d49271cc9381 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:41.812861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be2591-5a2d-4a03-b0e6-a84c36d9bec3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:41.816449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22b4c4-e746-4e95-85f5-6abc22e75e01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:41.817107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b4c4-e746-4e95-85f5-6abc22e75e01 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:41.817399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b4c4-e746-4e95-85f5-6abc22e75e01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:41.817675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b4c4-e746-4e95-85f5-6abc22e75e01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:41.819488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-be22b4c4-e746-4e95-85f5-6abc22e75e01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:41.837322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423933f5-c6fa-437e-a85c-3718fabd1f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:41.837974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423933f5-c6fa-437e-a85c-3718fabd1f28 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:41.838260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423933f5-c6fa-437e-a85c-3718fabd1f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:41.838538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423933f5-c6fa-437e-a85c-3718fabd1f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:41.840311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-423933f5-c6fa-437e-a85c-3718fabd1f28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:41.878961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22b4c4-e746-4e95-85f5-6abc22e75e01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:41.880395 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22b4c4-e746-4e95-85f5-6abc22e75e01 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:41.890878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1065/2114] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:41.894152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6dbe0-ba07-4add-9328-811f3d95824d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:41.894152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6dbe0-ba07-4add-9328-811f3d95824d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:41.894152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6dbe0-ba07-4add-9328-811f3d95824d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:41.906979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423933f5-c6fa-437e-a85c-3718fabd1f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:41.908421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423933f5-c6fa-437e-a85c-3718fabd1f28 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:41.915810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d2d6dbe0-ba07-4add-9328-811f3d95824d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:41.917234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d2d6dbe0-ba07-4add-9328-811f3d95824d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:41.920621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1050/2115] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:14:41.926249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1066/2116] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:41.935398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be42a7c-7c28-41f2-b11c-17abcca89fba None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:41.935692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be42a7c-7c28-41f2-b11c-17abcca89fba None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:41.935977 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be42a7c-7c28-41f2-b11c-17abcca89fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:41.957074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6be42a7c-7c28-41f2-b11c-17abcca89fba None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:41.958427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6be42a7c-7c28-41f2-b11c-17abcca89fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:41.970187 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1051/2117] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:41.972266 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6dbe0-ba07-4add-9328-811f3d95824d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:41.984560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6dbe0-ba07-4add-9328-811f3d95824d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:41.990000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bb176-3142-499d-b7e2-b71ce9d25f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:41.990708 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bb176-3142-499d-b7e2-b71ce9d25f0f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:41.991155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bb176-3142-499d-b7e2-b71ce9d25f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:41.991563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bb176-3142-499d-b7e2-b71ce9d25f0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:41.992279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bb176-3142-499d-b7e2-b71ce9d25f0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:41.992587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bb176-3142-499d-b7e2-b71ce9d25f0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:41.994764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bb176-3142-499d-b7e2-b71ce9d25f0f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:42.007417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1067/2118] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:42.012806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be42a7c-7c28-41f2-b11c-17abcca89fba None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:42.022337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be42a7c-7c28-41f2-b11c-17abcca89fba None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:42.028904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022a85f-6c73-42aa-ac64-4849ee1302ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:42.029907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a85f-6c73-42aa-ac64-4849ee1302ae admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.030290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a85f-6c73-42aa-ac64-4849ee1302ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.030638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a85f-6c73-42aa-ac64-4849ee1302ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.031710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022a85f-6c73-42aa-ac64-4849ee1302ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:42.034904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022a85f-6c73-42aa-ac64-4849ee1302ae admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:42.118454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:42.126284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1052/2119] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:42 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:42.130189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bb176-3142-499d-b7e2-b71ce9d25f0f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:42.141024 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bb176-3142-499d-b7e2-b71ce9d25f0f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:42.202541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07de0ad-acca-4a9e-9cae-b92e8641a783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:42.203168 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07de0ad-acca-4a9e-9cae-b92e8641a783 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.203454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07de0ad-acca-4a9e-9cae-b92e8641a783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.203733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07de0ad-acca-4a9e-9cae-b92e8641a783 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.204418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07de0ad-acca-4a9e-9cae-b92e8641a783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:42.206410 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07de0ad-acca-4a9e-9cae-b92e8641a783 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:42.306169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1068/2120] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:42 2022] POST /identity/v3/users => generated 283 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:42.310665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022a85f-6c73-42aa-ac64-4849ee1302ae admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:42.321490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022a85f-6c73-42aa-ac64-4849ee1302ae admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:42.326600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6fea3-acce-4ef8-8f4e-5c5858a0d06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:42.327218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6fea3-acce-4ef8-8f4e-5c5858a0d06f admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.327501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6fea3-acce-4ef8-8f4e-5c5858a0d06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.327774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6fea3-acce-4ef8-8f4e-5c5858a0d06f admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.328585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6fea3-acce-4ef8-8f4e-5c5858a0d06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:42.330263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6fea3-acce-4ef8-8f4e-5c5858a0d06f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:42.342794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1053/2121] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:42.345659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07de0ad-acca-4a9e-9cae-b92e8641a783 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:42.356502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07de0ad-acca-4a9e-9cae-b92e8641a783 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:42.361302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46c424-7904-444b-a8c8-27cedde84f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:42.365813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46c424-7904-444b-a8c8-27cedde84f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.365813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46c424-7904-444b-a8c8-27cedde84f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.365813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46c424-7904-444b-a8c8-27cedde84f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.365813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46c424-7904-444b-a8c8-27cedde84f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:42.365813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46c424-7904-444b-a8c8-27cedde84f8f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:42.378106 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1069/2122] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:14:42.382427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6fea3-acce-4ef8-8f4e-5c5858a0d06f admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:42.392656 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6fea3-acce-4ef8-8f4e-5c5858a0d06f admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:42.397377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83083430-bdf7-45b9-b742-2a7b9a11f8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:42.398034 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83083430-bdf7-45b9-b742-2a7b9a11f8f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.398319 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83083430-bdf7-45b9-b742-2a7b9a11f8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.398596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83083430-bdf7-45b9-b742-2a7b9a11f8f8 admin admin] PATH_INFO: `/v3/projects/c3ea93df3b774790831eb1c353887a4d/users/9fdcb41617e04a068210eebb44ff7f1e/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.412716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83083430-bdf7-45b9-b742-2a7b9a11f8f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3ea93df3b774790831eb1c353887a4d, user_id=9fdcb41617e04a068210eebb44ff7f1e, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:42.415408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83083430-bdf7-45b9-b742-2a7b9a11f8f8 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:42.487125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46c424-7904-444b-a8c8-27cedde84f8f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:42.488921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1f46c424-7904-444b-a8c8-27cedde84f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:42.499862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1054/2123] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:42 2022] PUT /identity/v3/projects/c3ea93df3b774790831eb1c353887a4d/users/9fdcb41617e04a068210eebb44ff7f1e/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:42.511229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3782cc-3caf-43b8-bf25-0551e000657a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.511561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3782cc-3caf-43b8-bf25-0551e000657a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.511787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3782cc-3caf-43b8-bf25-0551e000657a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.516957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46c424-7904-444b-a8c8-27cedde84f8f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:42.539597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7f3782cc-3caf-43b8-bf25-0551e000657a None None] MFA Rules not processed for user `9fdcb41617e04a068210eebb44ff7f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:42.541458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7f3782cc-3caf-43b8-bf25-0551e000657a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:42.562232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:42.597407 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:42.598428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.598877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.599154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] PATH_INFO: `/v3/projects/e048b2ccc0554f30b5180dc0955f0ff3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.600356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e048b2ccc0554f30b5180dc0955f0ff3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:42.602307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:42.620193 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1055/2124] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:42.624756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3782cc-3caf-43b8-bf25-0551e000657a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:42.646776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3782cc-3caf-43b8-bf25-0551e000657a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:42.651932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785995a4-7182-439f-9a9c-7ad3920ae68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:42.652584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785995a4-7182-439f-9a9c-7ad3920ae68b admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.652859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785995a4-7182-439f-9a9c-7ad3920ae68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.653164 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785995a4-7182-439f-9a9c-7ad3920ae68b admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.653835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785995a4-7182-439f-9a9c-7ad3920ae68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:42.655744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785995a4-7182-439f-9a9c-7ad3920ae68b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:42.711901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e048b2ccc0554f30b5180dc0955f0ff3', 'request_id': 'req-9da98163-8453-4c5f-9717-e50e371c5d1b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:42.718818 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e048b2ccc0554f30b5180dc0955f0ff3', 'request_id': 'req-9da98163-8453-4c5f-9717-e50e371c5d1b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:42.725573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e048b2ccc0554f30b5180dc0955f0ff3', 'request_id': 'req-9da98163-8453-4c5f-9717-e50e371c5d1b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:42.813916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1056/2125] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:42 2022] POST /identity/v3/projects => generated 363 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:42.815339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785995a4-7182-439f-9a9c-7ad3920ae68b admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:42.847312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] The token cache is being invalidate because project e048b2ccc0554f30b5180dc0955f0ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:42.849445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:42.851066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9da98163-8453-4c5f-9717-e50e371c5d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:42.858213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1070/2126] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:42 2022] DELETE /identity/v3/projects/e048b2ccc0554f30b5180dc0955f0ff3 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:42.863206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fe1d9-e078-46af-bef4-bc7540939f8a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.863499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fe1d9-e078-46af-bef4-bc7540939f8a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.864488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fe1d9-e078-46af-bef4-bc7540939f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.889242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6e1fe1d9-e078-46af-bef4-bc7540939f8a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:42.890619 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6e1fe1d9-e078-46af-bef4-bc7540939f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:42.891005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785995a4-7182-439f-9a9c-7ad3920ae68b admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:42.896191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe6f3c-f448-4fdd-af0e-ba5682780359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:42.896931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe6f3c-f448-4fdd-af0e-ba5682780359 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.897377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe6f3c-f448-4fdd-af0e-ba5682780359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.897756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe6f3c-f448-4fdd-af0e-ba5682780359 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.898532 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe6f3c-f448-4fdd-af0e-ba5682780359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:42.905990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1071/2127] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:42.906195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe6f3c-f448-4fdd-af0e-ba5682780359 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:42.908160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad2100-dd37-4c29-a2b8-9dfb4e033a68 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.908518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad2100-dd37-4c29-a2b8-9dfb4e033a68 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.909015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad2100-dd37-4c29-a2b8-9dfb4e033a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.930111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-48ad2100-dd37-4c29-a2b8-9dfb4e033a68 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:42.931354 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-48ad2100-dd37-4c29-a2b8-9dfb4e033a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:42.940055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1072/2128] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:42.944046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad2100-dd37-4c29-a2b8-9dfb4e033a68 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:42.953487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad2100-dd37-4c29-a2b8-9dfb4e033a68 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:42.958358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d9ad5-4774-4553-ac64-1a2cd83ef3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:42.958990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d9ad5-4774-4553-ac64-1a2cd83ef3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.959271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d9ad5-4774-4553-ac64-1a2cd83ef3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.959572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d9ad5-4774-4553-ac64-1a2cd83ef3a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.960269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d9ad5-4774-4553-ac64-1a2cd83ef3a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:42.960575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d9ad5-4774-4553-ac64-1a2cd83ef3a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:42.962256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d9ad5-4774-4553-ac64-1a2cd83ef3a0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:42.974097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1073/2129] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:14:42.978981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1057/2130] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:42 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:42.980600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d9ad5-4774-4553-ac64-1a2cd83ef3a0 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:42.980721 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe6f3c-f448-4fdd-af0e-ba5682780359 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:42.982371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cfbe6f3c-f448-4fdd-af0e-ba5682780359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:42.990511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d9ad5-4774-4553-ac64-1a2cd83ef3a0 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:42.996323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe6f3c-f448-4fdd-af0e-ba5682780359 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:42.996470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e79b9-06fb-4024-994a-fa46eadf2c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:42.996640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e79b9-06fb-4024-994a-fa46eadf2c03 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:42.996925 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e79b9-06fb-4024-994a-fa46eadf2c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:42.997234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e79b9-06fb-4024-994a-fa46eadf2c03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:42.997911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e79b9-06fb-4024-994a-fa46eadf2c03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:42.999808 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e79b9-06fb-4024-994a-fa46eadf2c03 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.000959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69d5bd-faf1-4640-9814-54cdaa813bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:43.001620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69d5bd-faf1-4640-9814-54cdaa813bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.001908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69d5bd-faf1-4640-9814-54cdaa813bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.002194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69d5bd-faf1-4640-9814-54cdaa813bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.002929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69d5bd-faf1-4640-9814-54cdaa813bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:43.004557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69d5bd-faf1-4640-9814-54cdaa813bd4 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.015401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1058/2131] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:43.019575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69d5bd-faf1-4640-9814-54cdaa813bd4 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:43.029278 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69d5bd-faf1-4640-9814-54cdaa813bd4 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:43.033989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2b892-021d-408d-8452-559d9d13a4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:43.034631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2b892-021d-408d-8452-559d9d13a4fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.034915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2b892-021d-408d-8452-559d9d13a4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.035196 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2b892-021d-408d-8452-559d9d13a4fb admin admin] PATH_INFO: `/v3/projects/67661e12675b419a902ece569bc81f7c/users/6534b84a825d41588ed9ccbd16cefd91/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.044213 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:43.050474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2b892-021d-408d-8452-559d9d13a4fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=67661e12675b419a902ece569bc81f7c, user_id=6534b84a825d41588ed9ccbd16cefd91, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:43.053202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2b892-021d-408d-8452-559d9d13a4fb admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.101044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1074/2132] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:42 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:43.105688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e79b9-06fb-4024-994a-fa46eadf2c03 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:43.125388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1059/2133] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:43 2022] PUT /identity/v3/projects/67661e12675b419a902ece569bc81f7c/users/6534b84a825d41588ed9ccbd16cefd91/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:43.129376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2b892-021d-408d-8452-559d9d13a4fb admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:43.178848 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e79b9-06fb-4024-994a-fa46eadf2c03 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:43.197647 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2b892-021d-408d-8452-559d9d13a4fb admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:43.202571 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f0daa-d509-4d41-808d-861fcd187fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:43.203311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f0daa-d509-4d41-808d-861fcd187fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.203702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f0daa-d509-4d41-808d-861fcd187fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.204200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f0daa-d509-4d41-808d-861fcd187fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.205013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f0daa-d509-4d41-808d-861fcd187fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:43.206709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f0daa-d509-4d41-808d-861fcd187fdd admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.210784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:43.220693 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1060/2134] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:14:43.221972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f0daa-d509-4d41-808d-861fcd187fdd admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:43.234995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f0daa-d509-4d41-808d-861fcd187fdd admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:43.240436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a3c58-ba78-4ff2-83d1-0b39acc69ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:43.241315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4bde2-e73c-44c8-a1d5-d6ec97ce966d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:43.241447 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a3c58-ba78-4ff2-83d1-0b39acc69ca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.242047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4bde2-e73c-44c8-a1d5-d6ec97ce966d admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.242175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a3c58-ba78-4ff2-83d1-0b39acc69ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.242367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4bde2-e73c-44c8-a1d5-d6ec97ce966d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.242457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a3c58-ba78-4ff2-83d1-0b39acc69ca7 admin admin] PATH_INFO: `/v3/projects/67661e12675b419a902ece569bc81f7c/users/6534b84a825d41588ed9ccbd16cefd91/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.242556 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4bde2-e73c-44c8-a1d5-d6ec97ce966d admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.243247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4bde2-e73c-44c8-a1d5-d6ec97ce966d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:43.245289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a3c58-ba78-4ff2-83d1-0b39acc69ca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67661e12675b419a902ece569bc81f7c, user_id=6534b84a825d41588ed9ccbd16cefd91, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:43.246029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4bde2-e73c-44c8-a1d5-d6ec97ce966d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.247938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a3c58-ba78-4ff2-83d1-0b39acc69ca7 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.259613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1061/2135] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:43 2022] PUT /identity/v3/projects/67661e12675b419a902ece569bc81f7c/users/6534b84a825d41588ed9ccbd16cefd91/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:43.273877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c911c-8ee1-4881-b5af-b8aeeb64a104 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.273877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c911c-8ee1-4881-b5af-b8aeeb64a104 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.274113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c911c-8ee1-4881-b5af-b8aeeb64a104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.304043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-b48c911c-8ee1-4881-b5af-b8aeeb64a104 None None] MFA Rules not processed for user `6534b84a825d41588ed9ccbd16cefd91`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:43.305313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b48c911c-8ee1-4881-b5af-b8aeeb64a104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:43.319901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1075/2136] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:43 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:43.325085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4bde2-e73c-44c8-a1d5-d6ec97ce966d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:43.337602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4bde2-e73c-44c8-a1d5-d6ec97ce966d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:43.382210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1062/2137] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:43 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:43.393942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c911c-8ee1-4881-b5af-b8aeeb64a104 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:43.395326 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b48c911c-8ee1-4881-b5af-b8aeeb64a104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:43.398736 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dd810-15cb-4a71-aeaf-828798173342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:43.399460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dd810-15cb-4a71-aeaf-828798173342 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.399812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dd810-15cb-4a71-aeaf-828798173342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.400675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dd810-15cb-4a71-aeaf-828798173342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.401438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dd810-15cb-4a71-aeaf-828798173342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:43.403069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dd810-15cb-4a71-aeaf-828798173342 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.416906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1076/2138] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:14:43.421938 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dd810-15cb-4a71-aeaf-828798173342 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:43.435773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dd810-15cb-4a71-aeaf-828798173342 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:43.441601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec52def-b731-467d-930c-608be440cff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:43.442273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec52def-b731-467d-930c-608be440cff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.442542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec52def-b731-467d-930c-608be440cff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.442820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec52def-b731-467d-930c-608be440cff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.443527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec52def-b731-467d-930c-608be440cff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:43.445160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec52def-b731-467d-930c-608be440cff0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.455523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1077/2139] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:43.460858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec52def-b731-467d-930c-608be440cff0 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:43.463373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c911c-8ee1-4881-b5af-b8aeeb64a104 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:43.467420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c7df5-4642-4b61-9cdc-f787612e67ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:43.468037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c7df5-4642-4b61-9cdc-f787612e67ab service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.468330 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c7df5-4642-4b61-9cdc-f787612e67ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.468601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c7df5-4642-4b61-9cdc-f787612e67ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.472138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec52def-b731-467d-930c-608be440cff0 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:43.476869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd7e4a-3689-4193-bfff-1552eb9f44de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:43.477542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd7e4a-3689-4193-bfff-1552eb9f44de admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.477824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd7e4a-3689-4193-bfff-1552eb9f44de admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.478050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c7df5-4642-4b61-9cdc-f787612e67ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:43.478153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd7e4a-3689-4193-bfff-1552eb9f44de admin admin] PATH_INFO: `/v3/projects/b05c462036494fd5840de730fdfcb7d7/users/1720f17c471e48d5a35bc2038129009f/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.480061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c7df5-4642-4b61-9cdc-f787612e67ab service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.484195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:43.491368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1063/2140] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:43 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:14:43.494679 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd7e4a-3689-4193-bfff-1552eb9f44de admin admin] RBAC: Authorizing `identity:create_grant(project_id=b05c462036494fd5840de730fdfcb7d7, user_id=1720f17c471e48d5a35bc2038129009f, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:43.497586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd7e4a-3689-4193-bfff-1552eb9f44de admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.569529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1078/2141] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:43 2022] PUT /identity/v3/projects/b05c462036494fd5840de730fdfcb7d7/users/1720f17c471e48d5a35bc2038129009f/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:43.580182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257ab91-debc-404b-85c4-a84fe8f0c0b6 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.580470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257ab91-debc-404b-85c4-a84fe8f0c0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.580753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257ab91-debc-404b-85c4-a84fe8f0c0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.611344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5257ab91-debc-404b-85c4-a84fe8f0c0b6 None None] MFA Rules not processed for user `1720f17c471e48d5a35bc2038129009f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:43.612608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5257ab91-debc-404b-85c4-a84fe8f0c0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:43.687079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1064/2142] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:43.700515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd7e4a-3689-4193-bfff-1552eb9f44de admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:43.700727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257ab91-debc-404b-85c4-a84fe8f0c0b6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:43.700916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e5bd7e4a-3689-4193-bfff-1552eb9f44de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:43.701035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5257ab91-debc-404b-85c4-a84fe8f0c0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:43.758065 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257ab91-debc-404b-85c4-a84fe8f0c0b6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:43.758671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd7e4a-3689-4193-bfff-1552eb9f44de admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:43.762491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a708d8-98ca-4132-9f04-6b0261e7a609 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47f1552e73648ceb8a421c0543c2974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:43.763238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a708d8-98ca-4132-9f04-6b0261e7a609 service glance] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.763373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ef42d-14d9-4b29-bf3d-8d9ee3b7cc78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:43.763542 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a708d8-98ca-4132-9f04-6b0261e7a609 service glance] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.766509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a708d8-98ca-4132-9f04-6b0261e7a609 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.766646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ef42d-14d9-4b29-bf3d-8d9ee3b7cc78 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:43.766646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ef42d-14d9-4b29-bf3d-8d9ee3b7cc78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:43.766646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ef42d-14d9-4b29-bf3d-8d9ee3b7cc78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:43.766646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b72ef42d-14d9-4b29-bf3d-8d9ee3b7cc78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:43.772623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a708d8-98ca-4132-9f04-6b0261e7a609 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:43.774136 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a708d8-98ca-4132-9f04-6b0261e7a609 service glance] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.783923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1065/2143] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:43.810605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:43.833822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ef42d-14d9-4b29-bf3d-8d9ee3b7cc78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:43.835526 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ef42d-14d9-4b29-bf3d-8d9ee3b7cc78 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:43.846667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1079/2144] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:47.631121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a708d8-98ca-4132-9f04-6b0261e7a609 service glance] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:47.704308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a708d8-98ca-4132-9f04-6b0261e7a609 service glance] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:47.709090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:47.709806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:47.710165 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:47.710449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] PATH_INFO: `/v3/users/9fdcb41617e04a068210eebb44ff7f1e` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:47.711628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fdcb41617e04a068210eebb44ff7f1e)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:47.714187 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:47.891841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:47.905644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fdcb41617e04a068210eebb44ff7f1e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:47.913757 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fdcb41617e04a068210eebb44ff7f1e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:47.997713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fdcb41617e04a068210eebb44ff7f1e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:47.998406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fdcb41617e04a068210eebb44ff7f1e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:48.011981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fdcb41617e04a068210eebb44ff7f1e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:48.018051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1066/2145] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:47 2022] DELETE /identity/v3/users/9fdcb41617e04a068210eebb44ff7f1e => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:48.022488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f1f45-1f3c-42b9-a65d-4369ddddb971 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:48.022945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f1f45-1f3c-42b9-a65d-4369ddddb971 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:48.023350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f1f45-1f3c-42b9-a65d-4369ddddb971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:48.055557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-792f1f45-1f3c-42b9-a65d-4369ddddb971 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:48.057364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-792f1f45-1f3c-42b9-a65d-4369ddddb971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:48.123899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1080/2146] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:48.131318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:48.132800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:48.194891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d9b00-ddf7-4285-9937-738af25a3384 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:48.199407 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2627c2-ca53-4167-acc3-f130d777d3f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:48.200111 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2627c2-ca53-4167-acc3-f130d777d3f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:48.200393 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2627c2-ca53-4167-acc3-f130d777d3f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:48.200665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2627c2-ca53-4167-acc3-f130d777d3f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:48.210040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2627c2-ca53-4167-acc3-f130d777d3f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:48.211469 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2627c2-ca53-4167-acc3-f130d777d3f2 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:48.224657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1067/2147] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:48.668095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f1f45-1f3c-42b9-a65d-4369ddddb971 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:48.669972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-792f1f45-1f3c-42b9-a65d-4369ddddb971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:48.688525 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f1f45-1f3c-42b9-a65d-4369ddddb971 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:48.693625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:48.694293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:48.694681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:48.694835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] PATH_INFO: `/v3/projects/c3ea93df3b774790831eb1c353887a4d` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:48.696193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3ea93df3b774790831eb1c353887a4d)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:48.698475 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:48.875091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3ea93df3b774790831eb1c353887a4d', 'request_id': 'req-0981286b-a5dd-4353-84ff-1c683d0a1fa3'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:48.882436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3ea93df3b774790831eb1c353887a4d', 'request_id': 'req-0981286b-a5dd-4353-84ff-1c683d0a1fa3'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:48.892415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3ea93df3b774790831eb1c353887a4d', 'request_id': 'req-0981286b-a5dd-4353-84ff-1c683d0a1fa3'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:49.010282 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] The token cache is being invalidate because project c3ea93df3b774790831eb1c353887a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:49.012943 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:49.012943 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:49.013341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1081/2148] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:48 2022] DELETE /identity/v3/projects/c3ea93df3b774790831eb1c353887a4d => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:49.016995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2627c2-ca53-4167-acc3-f130d777d3f2 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:49.027628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2627c2-ca53-4167-acc3-f130d777d3f2 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:49.076621 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:49.078071 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:49.091478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de8bb4-70da-46a0-b828-b8d1ecc15ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:49.092130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de8bb4-70da-46a0-b828-b8d1ecc15ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:49.093243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de8bb4-70da-46a0-b828-b8d1ecc15ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:49.093243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de8bb4-70da-46a0-b828-b8d1ecc15ea8 admin admin] PATH_INFO: `/v3/users/6534b84a825d41588ed9ccbd16cefd91` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:49.094052 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de8bb4-70da-46a0-b828-b8d1ecc15ea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6534b84a825d41588ed9ccbd16cefd91)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:49.095978 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de8bb4-70da-46a0-b828-b8d1ecc15ea8 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:49.139365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981286b-a5dd-4353-84ff-1c683d0a1fa3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:49.143150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa973ba9-502a-4935-8732-7cdc17f8a446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:49.143773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973ba9-502a-4935-8732-7cdc17f8a446 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:49.144055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973ba9-502a-4935-8732-7cdc17f8a446 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:49.144335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973ba9-502a-4935-8732-7cdc17f8a446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:49.146175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fa973ba9-502a-4935-8732-7cdc17f8a446 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:49.160014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:49.206891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa973ba9-502a-4935-8732-7cdc17f8a446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:49.208435 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa973ba9-502a-4935-8732-7cdc17f8a446 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:49.218729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1082/2149] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:14:49.345545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-95de8bb4-70da-46a0-b828-b8d1ecc15ea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6534b84a825d41588ed9ccbd16cefd91'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:49.350894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:49.353022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-95de8bb4-70da-46a0-b828-b8d1ecc15ea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6534b84a825d41588ed9ccbd16cefd91'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:49.432911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-95de8bb4-70da-46a0-b828-b8d1ecc15ea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6534b84a825d41588ed9ccbd16cefd91'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:49.433574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-95de8bb4-70da-46a0-b828-b8d1ecc15ea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6534b84a825d41588ed9ccbd16cefd91'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:49.449134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-95de8bb4-70da-46a0-b828-b8d1ecc15ea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6534b84a825d41588ed9ccbd16cefd91'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:49.462333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1068/2150] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:49 2022] DELETE /identity/v3/users/6534b84a825d41588ed9ccbd16cefd91 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:49.902989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa973ba9-502a-4935-8732-7cdc17f8a446 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:49.904503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fa973ba9-502a-4935-8732-7cdc17f8a446 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:49.933191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa973ba9-502a-4935-8732-7cdc17f8a446 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:50.033171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:50.033858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:50.034219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:50.034510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] PATH_INFO: `/v3/projects/67661e12675b419a902ece569bc81f7c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:50.035735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67661e12675b419a902ece569bc81f7c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:50.038235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:50.146782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67661e12675b419a902ece569bc81f7c', 'request_id': 'req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:50.152836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67661e12675b419a902ece569bc81f7c', 'request_id': 'req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:50.159172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67661e12675b419a902ece569bc81f7c', 'request_id': 'req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:50.237853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] The token cache is being invalidate because project 67661e12675b419a902ece569bc81f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:50.238170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:50.238845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a3cb4a39-f9e9-4530-9990-09fdd1865cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:50.240381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1083/2151] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:49 2022] DELETE /identity/v3/projects/67661e12675b419a902ece569bc81f7c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:50.245738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c168a6-bd46-4476-a6bb-2289e68562fb None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:50.246294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c168a6-bd46-4476-a6bb-2289e68562fb None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:50.246294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c168a6-bd46-4476-a6bb-2289e68562fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:50.266895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-49c168a6-bd46-4476-a6bb-2289e68562fb None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:50.268178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-49c168a6-bd46-4476-a6bb-2289e68562fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:50.330101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1069/2152] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:50.338606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2a6ca-59c2-4186-b23d-9ac3ecfc5ddb None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:50.338983 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2a6ca-59c2-4186-b23d-9ac3ecfc5ddb None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:50.339117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2a6ca-59c2-4186-b23d-9ac3ecfc5ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:50.360733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ade2a6ca-59c2-4186-b23d-9ac3ecfc5ddb None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:50.361993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ade2a6ca-59c2-4186-b23d-9ac3ecfc5ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:50.370080 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1084/2153] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:50.374441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c168a6-bd46-4476-a6bb-2289e68562fb None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:50.384483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c168a6-bd46-4476-a6bb-2289e68562fb None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:50.389405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ec008-8964-49e0-9fb1-2089ce932714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:50.390027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec008-8964-49e0-9fb1-2089ce932714 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:50.390313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec008-8964-49e0-9fb1-2089ce932714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:50.390592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec008-8964-49e0-9fb1-2089ce932714 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:50.391297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ec008-8964-49e0-9fb1-2089ce932714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:50.391607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ec008-8964-49e0-9fb1-2089ce932714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:50.393266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ec008-8964-49e0-9fb1-2089ce932714 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:50.405515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1070/2154] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:50.409865 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2a6ca-59c2-4186-b23d-9ac3ecfc5ddb None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:50.419959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2a6ca-59c2-4186-b23d-9ac3ecfc5ddb None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:50.424860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae6dac-7e34-4f60-8e63-d25ed2e6d4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:50.425442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae6dac-7e34-4f60-8e63-d25ed2e6d4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:50.425726 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae6dac-7e34-4f60-8e63-d25ed2e6d4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:50.426006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae6dac-7e34-4f60-8e63-d25ed2e6d4a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:50.426723 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae6dac-7e34-4f60-8e63-d25ed2e6d4a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:50.428686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae6dac-7e34-4f60-8e63-d25ed2e6d4a7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:50.567352 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1085/2155] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:50 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:50.570140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ec008-8964-49e0-9fb1-2089ce932714 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:50.581038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ec008-8964-49e0-9fb1-2089ce932714 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:50.647388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b24c9-0f0e-412b-8bc9-2ebf6ddd9080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:50.648202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b24c9-0f0e-412b-8bc9-2ebf6ddd9080 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:50.648563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b24c9-0f0e-412b-8bc9-2ebf6ddd9080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:50.649119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b24c9-0f0e-412b-8bc9-2ebf6ddd9080 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:50.649670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b24c9-0f0e-412b-8bc9-2ebf6ddd9080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:50.651688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b24c9-0f0e-412b-8bc9-2ebf6ddd9080 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:50.752861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1071/2156] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:50 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:50.757259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae6dac-7e34-4f60-8e63-d25ed2e6d4a7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:50.767859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae6dac-7e34-4f60-8e63-d25ed2e6d4a7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:50.773041 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6b55a-c2c0-463f-a2d0-757be1a996e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:50.774190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6b55a-c2c0-463f-a2d0-757be1a996e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:50.774488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6b55a-c2c0-463f-a2d0-757be1a996e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:50.774766 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6b55a-c2c0-463f-a2d0-757be1a996e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:50.775958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6b55a-c2c0-463f-a2d0-757be1a996e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:50.777750 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6b55a-c2c0-463f-a2d0-757be1a996e8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:50.788958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1086/2157] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:50.793903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b24c9-0f0e-412b-8bc9-2ebf6ddd9080 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:50.805515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b24c9-0f0e-412b-8bc9-2ebf6ddd9080 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:50.810538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6880132-2105-49e2-91a3-bf07c33b9274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:50.811159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6880132-2105-49e2-91a3-bf07c33b9274 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:50.811445 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6880132-2105-49e2-91a3-bf07c33b9274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:50.811724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6880132-2105-49e2-91a3-bf07c33b9274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:50.812428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6880132-2105-49e2-91a3-bf07c33b9274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:50.814124 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6880132-2105-49e2-91a3-bf07c33b9274 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:50.824890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1072/2158] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:50.829028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6b55a-c2c0-463f-a2d0-757be1a996e8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:50.835593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:50.843205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6b55a-c2c0-463f-a2d0-757be1a996e8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:50.845342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32261d9-b36a-4ce4-a80f-0e470f4b0e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:50.846009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32261d9-b36a-4ce4-a80f-0e470f4b0e87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:50.846294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32261d9-b36a-4ce4-a80f-0e470f4b0e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:50.846576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32261d9-b36a-4ce4-a80f-0e470f4b0e87 admin admin] PATH_INFO: `/v3/projects/d86f5d967df54762aefd8b5311f45dda/users/7253c33029f24d048490fb1d3083f879/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:50.862488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32261d9-b36a-4ce4-a80f-0e470f4b0e87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d86f5d967df54762aefd8b5311f45dda, user_id=7253c33029f24d048490fb1d3083f879, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:50.865273 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32261d9-b36a-4ce4-a80f-0e470f4b0e87 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:50.994835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1087/2159] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:50 2022] PUT /identity/v3/projects/d86f5d967df54762aefd8b5311f45dda/users/7253c33029f24d048490fb1d3083f879/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:51.006188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba284c54-5f35-4de4-bab1-2a76a3a567bd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:51.006487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba284c54-5f35-4de4-bab1-2a76a3a567bd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:51.006745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba284c54-5f35-4de4-bab1-2a76a3a567bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:51.037333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-ba284c54-5f35-4de4-bab1-2a76a3a567bd None None] MFA Rules not processed for user `7253c33029f24d048490fb1d3083f879`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:51.038744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ba284c54-5f35-4de4-bab1-2a76a3a567bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:51.076106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:51.123097 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1073/2160] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:51.124243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32261d9-b36a-4ce4-a80f-0e470f4b0e87 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:51.233587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32261d9-b36a-4ce4-a80f-0e470f4b0e87 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:51.239355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a08411-525a-42bc-bcc8-8d0841adc702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:51.242762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08411-525a-42bc-bcc8-8d0841adc702 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:51.242762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08411-525a-42bc-bcc8-8d0841adc702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:51.242762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a08411-525a-42bc-bcc8-8d0841adc702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:51.242762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a08411-525a-42bc-bcc8-8d0841adc702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:51.244281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a08411-525a-42bc-bcc8-8d0841adc702 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:51.330400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1088/2161] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:51 2022] POST /identity/v3/projects => generated 375 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:51.334245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba284c54-5f35-4de4-bab1-2a76a3a567bd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:51.345968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba284c54-5f35-4de4-bab1-2a76a3a567bd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:51.409715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08492c2-b6ef-4abb-9cc7-d2b06480e882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:51.410340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08492c2-b6ef-4abb-9cc7-d2b06480e882 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:51.410629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08492c2-b6ef-4abb-9cc7-d2b06480e882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:51.410909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08492c2-b6ef-4abb-9cc7-d2b06480e882 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:51.411600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08492c2-b6ef-4abb-9cc7-d2b06480e882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:51.413536 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08492c2-b6ef-4abb-9cc7-d2b06480e882 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:51.502553 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1074/2162] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:51 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:14:51.506512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a08411-525a-42bc-bcc8-8d0841adc702 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:51.518442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a08411-525a-42bc-bcc8-8d0841adc702 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:51.523254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7e90a-b1d9-491d-8bc2-a099555d14b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:51.523871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7e90a-b1d9-491d-8bc2-a099555d14b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:51.524150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7e90a-b1d9-491d-8bc2-a099555d14b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:51.524425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7e90a-b1d9-491d-8bc2-a099555d14b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:51.525160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7e90a-b1d9-491d-8bc2-a099555d14b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:51.526796 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7e90a-b1d9-491d-8bc2-a099555d14b8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:51.537576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1089/2163] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:51.541929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08492c2-b6ef-4abb-9cc7-d2b06480e882 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:51.552255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08492c2-b6ef-4abb-9cc7-d2b06480e882 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:51.556930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7b91c-4c97-47e2-aee0-db5c004040cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:51.557684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7b91c-4c97-47e2-aee0-db5c004040cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:51.558537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7b91c-4c97-47e2-aee0-db5c004040cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:51.558893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7b91c-4c97-47e2-aee0-db5c004040cb admin admin] PATH_INFO: `/v3/projects/4f3fb6f4f2294a68be22107af07a5e9a/users/8184af003d7c4bcb82d4685e3ef66941/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:51.574767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7b91c-4c97-47e2-aee0-db5c004040cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3fb6f4f2294a68be22107af07a5e9a, user_id=8184af003d7c4bcb82d4685e3ef66941, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:51.577636 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7b91c-4c97-47e2-aee0-db5c004040cb admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:51.684763 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1075/2164] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:51 2022] PUT /identity/v3/projects/4f3fb6f4f2294a68be22107af07a5e9a/users/8184af003d7c4bcb82d4685e3ef66941/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:51.688979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7e90a-b1d9-491d-8bc2-a099555d14b8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:51.700570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7e90a-b1d9-491d-8bc2-a099555d14b8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:51.762058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab759647-f3de-4848-9437-59e98fb3987a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:51.762699 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab759647-f3de-4848-9437-59e98fb3987a admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:51.762989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab759647-f3de-4848-9437-59e98fb3987a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:51.763416 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab759647-f3de-4848-9437-59e98fb3987a admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:51.764184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab759647-f3de-4848-9437-59e98fb3987a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:51.765820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab759647-f3de-4848-9437-59e98fb3987a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:51.776991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1090/2165] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:51.781189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7b91c-4c97-47e2-aee0-db5c004040cb admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:51.793547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7b91c-4c97-47e2-aee0-db5c004040cb admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:51.798472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd469cc-5ebb-40ae-b3fd-142361a93621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:51.799227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd469cc-5ebb-40ae-b3fd-142361a93621 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:51.799517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd469cc-5ebb-40ae-b3fd-142361a93621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:51.799804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd469cc-5ebb-40ae-b3fd-142361a93621 admin admin] PATH_INFO: `/v3/projects/4f3fb6f4f2294a68be22107af07a5e9a/users/8184af003d7c4bcb82d4685e3ef66941/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:51.801791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd469cc-5ebb-40ae-b3fd-142361a93621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3fb6f4f2294a68be22107af07a5e9a, user_id=8184af003d7c4bcb82d4685e3ef66941, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:51.804465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd469cc-5ebb-40ae-b3fd-142361a93621 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:51.815431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1076/2166] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:51 2022] PUT /identity/v3/projects/4f3fb6f4f2294a68be22107af07a5e9a/users/8184af003d7c4bcb82d4685e3ef66941/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:51.826358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03354d11-dd52-4de1-8c27-6050d17faf5a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:51.826646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03354d11-dd52-4de1-8c27-6050d17faf5a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:51.826923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03354d11-dd52-4de1-8c27-6050d17faf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:51.854712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-03354d11-dd52-4de1-8c27-6050d17faf5a None None] MFA Rules not processed for user `8184af003d7c4bcb82d4685e3ef66941`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:51.855994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-03354d11-dd52-4de1-8c27-6050d17faf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:51.928036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1091/2167] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:51.939294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd469cc-5ebb-40ae-b3fd-142361a93621 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:51.941012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-dcd469cc-5ebb-40ae-b3fd-142361a93621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:52.002235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd469cc-5ebb-40ae-b3fd-142361a93621 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:52.004178 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:52.006019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc606fd4-48ef-491c-bc56-f51b819609bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:52.006634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606fd4-48ef-491c-bc56-f51b819609bd service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:52.006950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606fd4-48ef-491c-bc56-f51b819609bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:52.007222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606fd4-48ef-491c-bc56-f51b819609bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:52.063424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc606fd4-48ef-491c-bc56-f51b819609bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:52.064872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc606fd4-48ef-491c-bc56-f51b819609bd service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:52.075059 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1077/2168] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:52.122053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:53.657419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03354d11-dd52-4de1-8c27-6050d17faf5a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:53.660352 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-03354d11-dd52-4de1-8c27-6050d17faf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:53.745751 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03354d11-dd52-4de1-8c27-6050d17faf5a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:53.756513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:53.756513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:53.756513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:53.756513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] PATH_INFO: `/v3/users/5deabdca4b8943b1843a2203747a0024` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:53.758030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5deabdca4b8943b1843a2203747a0024)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:53.760130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:54.025396 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5deabdca4b8943b1843a2203747a0024'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:54.026716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5deabdca4b8943b1843a2203747a0024'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:54.052317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5deabdca4b8943b1843a2203747a0024'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:54.147851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5deabdca4b8943b1843a2203747a0024'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:54.148463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5deabdca4b8943b1843a2203747a0024'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:54.190601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1092/2169] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:53 2022] DELETE /identity/v3/users/5deabdca4b8943b1843a2203747a0024 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:54.195792 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2867b99-1cc3-4aa1-8ced-619cedee54a0 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:54.196258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2867b99-1cc3-4aa1-8ced-619cedee54a0 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:54.196340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2867b99-1cc3-4aa1-8ced-619cedee54a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:54.226644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c2867b99-1cc3-4aa1-8ced-619cedee54a0 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:54.227881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c2867b99-1cc3-4aa1-8ced-619cedee54a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:54.305995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1078/2170] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:54.317189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:54.318738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:54.385502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38e701-3ef0-4c0b-8750-a82ccdde9605 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:54.389091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08661f06-9a2b-4b34-9a7d-7915804c28bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:54.389712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08661f06-9a2b-4b34-9a7d-7915804c28bb service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:54.389996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08661f06-9a2b-4b34-9a7d-7915804c28bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:54.390286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08661f06-9a2b-4b34-9a7d-7915804c28bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:54.399647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08661f06-9a2b-4b34-9a7d-7915804c28bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:54.401147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08661f06-9a2b-4b34-9a7d-7915804c28bb service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:54.405973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:54.412376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1093/2171] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:14:54.857362 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2867b99-1cc3-4aa1-8ced-619cedee54a0 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:54.859019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c2867b99-1cc3-4aa1-8ced-619cedee54a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:54.874565 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2867b99-1cc3-4aa1-8ced-619cedee54a0 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:54.880217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:54.880912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:54.881251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:54.881563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] PATH_INFO: `/v3/projects/fa94416b76ed4a8dbb7e6a15d8cf6e3a` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:54.882792 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa94416b76ed4a8dbb7e6a15d8cf6e3a)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:54.884881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:55.021808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa94416b76ed4a8dbb7e6a15d8cf6e3a', 'request_id': 'req-f6c719d4-99d7-4d28-a39f-f4624fe4800d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:55.029661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa94416b76ed4a8dbb7e6a15d8cf6e3a', 'request_id': 'req-f6c719d4-99d7-4d28-a39f-f4624fe4800d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:55.105858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa94416b76ed4a8dbb7e6a15d8cf6e3a', 'request_id': 'req-f6c719d4-99d7-4d28-a39f-f4624fe4800d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:55.117678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] The token cache is being invalidate because project fa94416b76ed4a8dbb7e6a15d8cf6e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:55.117966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:55.118612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:55.121634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1079/2172] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:54 2022] DELETE /identity/v3/projects/fa94416b76ed4a8dbb7e6a15d8cf6e3a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:55.125705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08661f06-9a2b-4b34-9a7d-7915804c28bb service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:55.135751 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08661f06-9a2b-4b34-9a7d-7915804c28bb service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:55.199979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:55.200647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:55.200935 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:55.201245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] PATH_INFO: `/v3/users/629137049b994cd5adf332a4da25a18d` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:55.202459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=629137049b994cd5adf332a4da25a18d)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:55.204457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:55.429380 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '629137049b994cd5adf332a4da25a18d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:55.430618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '629137049b994cd5adf332a4da25a18d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:55.460856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '629137049b994cd5adf332a4da25a18d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:55.552609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '629137049b994cd5adf332a4da25a18d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:55.553345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '629137049b994cd5adf332a4da25a18d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:55.572512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1094/2173] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:55 2022] DELETE /identity/v3/users/629137049b994cd5adf332a4da25a18d => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:55.988716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:55.990222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:56.004865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c719d4-99d7-4d28-a39f-f4624fe4800d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:56.057720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:56.063719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:56.064371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:56.064658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:56.064942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] PATH_INFO: `/v3/projects/118fdddd109e4a94933c1bbf58d2586a` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:56.066142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] RBAC: Authorizing `identity:delete_project(project_id=118fdddd109e4a94933c1bbf58d2586a)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:56.068063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:56.175507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '118fdddd109e4a94933c1bbf58d2586a', 'request_id': 'req-221d2761-e269-433a-b227-1f1a673f711d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:56.180857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '118fdddd109e4a94933c1bbf58d2586a', 'request_id': 'req-221d2761-e269-433a-b227-1f1a673f711d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:56.226049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '118fdddd109e4a94933c1bbf58d2586a', 'request_id': 'req-221d2761-e269-433a-b227-1f1a673f711d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:56.237727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] The token cache is being invalidate because project 118fdddd109e4a94933c1bbf58d2586a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:56.237958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:56.238665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:56.240189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1080/2174] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:55 2022] DELETE /identity/v3/projects/118fdddd109e4a94933c1bbf58d2586a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:56.244287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:56.245904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:56.258744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d212b94-7b6f-4ea7-8ee1-2346dcf51c62 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:56.317804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed0b51-c32e-4876-97ab-1056ff970971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:56.318454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed0b51-c32e-4876-97ab-1056ff970971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:56.318732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed0b51-c32e-4876-97ab-1056ff970971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:56.319004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed0b51-c32e-4876-97ab-1056ff970971 admin admin] PATH_INFO: `/v3/users/0886e639d68343e4b3fd506e2d98723e` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:56.320180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed0b51-c32e-4876-97ab-1056ff970971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0886e639d68343e4b3fd506e2d98723e)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:56.322396 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed0b51-c32e-4876-97ab-1056ff970971 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:56.346301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:56.482968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f6ed0b51-c32e-4876-97ab-1056ff970971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0886e639d68343e4b3fd506e2d98723e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:56.483735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f6ed0b51-c32e-4876-97ab-1056ff970971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0886e639d68343e4b3fd506e2d98723e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:56.493326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f6ed0b51-c32e-4876-97ab-1056ff970971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0886e639d68343e4b3fd506e2d98723e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:56.558207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f6ed0b51-c32e-4876-97ab-1056ff970971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0886e639d68343e4b3fd506e2d98723e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:56.558900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f6ed0b51-c32e-4876-97ab-1056ff970971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0886e639d68343e4b3fd506e2d98723e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:56.577756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1095/2175] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:14:56 2022] DELETE /identity/v3/users/0886e639d68343e4b3fd506e2d98723e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:56.970623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:56.972892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:57.001519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d2761-e269-433a-b227-1f1a673f711d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:57.067397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:57.067961 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:57.068249 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:57.068531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] PATH_INFO: `/v3/projects/f0d204a0ff1643f4b423dd4a9a1881b2` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:57.069754 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0d204a0ff1643f4b423dd4a9a1881b2)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:57.071688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:57.194422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0d204a0ff1643f4b423dd4a9a1881b2', 'request_id': 'req-758ed508-7d23-4ed1-aeda-7d97477ce41c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:57.200009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0d204a0ff1643f4b423dd4a9a1881b2', 'request_id': 'req-758ed508-7d23-4ed1-aeda-7d97477ce41c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:57.278198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0d204a0ff1643f4b423dd4a9a1881b2', 'request_id': 'req-758ed508-7d23-4ed1-aeda-7d97477ce41c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:57.290752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] The token cache is being invalidate because project f0d204a0ff1643f4b423dd4a9a1881b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:14:57.291432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:57.291736 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-758ed508-7d23-4ed1-aeda-7d97477ce41c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:14:57.294419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1081/2176] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:14:56 2022] DELETE /identity/v3/projects/f0d204a0ff1643f4b423dd4a9a1881b2 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:14:57.300733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c93451-4fff-41c0-9387-d31545b4108d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:57.301346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c93451-4fff-41c0-9387-d31545b4108d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:57.301346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c93451-4fff-41c0-9387-d31545b4108d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:57.323594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e6c93451-4fff-41c0-9387-d31545b4108d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:57.324768 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e6c93451-4fff-41c0-9387-d31545b4108d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:57.392560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1096/2177] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:57.400779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd302e58-9daf-4ece-abb6-713ad6fa4859 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:57.400921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd302e58-9daf-4ece-abb6-713ad6fa4859 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:57.401210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd302e58-9daf-4ece-abb6-713ad6fa4859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:57.426322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-dd302e58-9daf-4ece-abb6-713ad6fa4859 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:57.427570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-dd302e58-9daf-4ece-abb6-713ad6fa4859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:57.442141 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1082/2178] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:57.446437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c93451-4fff-41c0-9387-d31545b4108d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:57.456691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c93451-4fff-41c0-9387-d31545b4108d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:57.461353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b702b-6512-4d4a-add5-63cb9c86d774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:57.461959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b702b-6512-4d4a-add5-63cb9c86d774 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:57.462235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b702b-6512-4d4a-add5-63cb9c86d774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:57.462508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b702b-6512-4d4a-add5-63cb9c86d774 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:57.463450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b702b-6512-4d4a-add5-63cb9c86d774 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:14:57.463663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b702b-6512-4d4a-add5-63cb9c86d774 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:57.465323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b702b-6512-4d4a-add5-63cb9c86d774 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:57.486442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1097/2179] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:14:57.498940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd302e58-9daf-4ece-abb6-713ad6fa4859 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:57.508269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd302e58-9daf-4ece-abb6-713ad6fa4859 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:57.513086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31bf6f-39b3-4470-ba3c-60fa28f1d016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:57.513729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31bf6f-39b3-4470-ba3c-60fa28f1d016 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:57.514013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31bf6f-39b3-4470-ba3c-60fa28f1d016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:57.514287 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31bf6f-39b3-4470-ba3c-60fa28f1d016 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:57.514960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31bf6f-39b3-4470-ba3c-60fa28f1d016 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:57.516853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31bf6f-39b3-4470-ba3c-60fa28f1d016 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:57.526392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:57.655673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1083/2180] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:57 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:57.656953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b702b-6512-4d4a-add5-63cb9c86d774 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:57.669465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b702b-6512-4d4a-add5-63cb9c86d774 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:57.728147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd3828-0416-4d35-8321-6c950ca7a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:57.729250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd3828-0416-4d35-8321-6c950ca7a6ac admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:57.729537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd3828-0416-4d35-8321-6c950ca7a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:57.730541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd3828-0416-4d35-8321-6c950ca7a6ac admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:57.731218 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd3828-0416-4d35-8321-6c950ca7a6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:57.733166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd3828-0416-4d35-8321-6c950ca7a6ac admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:57.849444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1098/2181] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:57 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:57.858137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31bf6f-39b3-4470-ba3c-60fa28f1d016 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:57.868440 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31bf6f-39b3-4470-ba3c-60fa28f1d016 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:57.873613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be83916-928e-48ca-8893-04b420a23c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:57.874218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83916-928e-48ca-8893-04b420a23c1b admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:57.874501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83916-928e-48ca-8893-04b420a23c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:57.874772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83916-928e-48ca-8893-04b420a23c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:57.875466 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be83916-928e-48ca-8893-04b420a23c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:57.877136 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be83916-928e-48ca-8893-04b420a23c1b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:57.887850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1084/2182] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:57.891672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd3828-0416-4d35-8321-6c950ca7a6ac admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:57.902625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd3828-0416-4d35-8321-6c950ca7a6ac admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:57.907294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401d85f-e5be-43c5-99b1-fe0e61e84ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:57.907901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401d85f-e5be-43c5-99b1-fe0e61e84ebd admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:57.908186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401d85f-e5be-43c5-99b1-fe0e61e84ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:57.908463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401d85f-e5be-43c5-99b1-fe0e61e84ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:57.909196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401d85f-e5be-43c5-99b1-fe0e61e84ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:57.910782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401d85f-e5be-43c5-99b1-fe0e61e84ebd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:57.921703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1099/2183] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:57.925714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be83916-928e-48ca-8893-04b420a23c1b admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:57.935562 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be83916-928e-48ca-8893-04b420a23c1b admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:57.940043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527c5dc-4825-4794-a92a-d7acae83458a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:57.940698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527c5dc-4825-4794-a92a-d7acae83458a admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:57.940997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527c5dc-4825-4794-a92a-d7acae83458a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:57.941290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527c5dc-4825-4794-a92a-d7acae83458a admin admin] PATH_INFO: `/v3/projects/1e064b693a124655b12e8987a9948717/users/6f9c3013799c40eea114118a96c66740/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:57.956218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527c5dc-4825-4794-a92a-d7acae83458a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e064b693a124655b12e8987a9948717, user_id=6f9c3013799c40eea114118a96c66740, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:57.958888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527c5dc-4825-4794-a92a-d7acae83458a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:58.077515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1085/2184] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:57 2022] PUT /identity/v3/projects/1e064b693a124655b12e8987a9948717/users/6f9c3013799c40eea114118a96c66740/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:14:58.082617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26078f-8b04-4d24-b5fd-23f7f1883652 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:58.082852 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26078f-8b04-4d24-b5fd-23f7f1883652 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:58.083126 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26078f-8b04-4d24-b5fd-23f7f1883652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:58.090972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:14:58.102856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6c26078f-8b04-4d24-b5fd-23f7f1883652 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:58.104049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6c26078f-8b04-4d24-b5fd-23f7f1883652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:58.162221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1100/2185] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:14:58.171976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527c5dc-4825-4794-a92a-d7acae83458a admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:58.173771 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9527c5dc-4825-4794-a92a-d7acae83458a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:58.230421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527c5dc-4825-4794-a92a-d7acae83458a admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:58.234050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef9044-7410-48d9-be4c-d72f1e7d3a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:58.234660 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef9044-7410-48d9-be4c-d72f1e7d3a08 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:58.234943 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef9044-7410-48d9-be4c-d72f1e7d3a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:58.235217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef9044-7410-48d9-be4c-d72f1e7d3a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:58.244026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef9044-7410-48d9-be4c-d72f1e7d3a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:58.246083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef9044-7410-48d9-be4c-d72f1e7d3a08 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:58.257254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1086/2186] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:14:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:14:59.491878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce35fc6-0990-487e-9ef1-00a77ce6ff3e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:59.492344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce35fc6-0990-487e-9ef1-00a77ce6ff3e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:59.492450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce35fc6-0990-487e-9ef1-00a77ce6ff3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:59.523476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9ce35fc6-0990-487e-9ef1-00a77ce6ff3e None None] MFA Rules not processed for user `6f9c3013799c40eea114118a96c66740`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:14:59.529085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9ce35fc6-0990-487e-9ef1-00a77ce6ff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:14:59.594246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1101/2187] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:14:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:14:59.598732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef9044-7410-48d9-be4c-d72f1e7d3a08 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:59.609216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef9044-7410-48d9-be4c-d72f1e7d3a08 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:59.613809 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b910eee-a6a6-4dd2-a9dd-30b47fc83688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:59.614425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910eee-a6a6-4dd2-a9dd-30b47fc83688 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:59.614714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910eee-a6a6-4dd2-a9dd-30b47fc83688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:59.614995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910eee-a6a6-4dd2-a9dd-30b47fc83688 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:59.615666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b910eee-a6a6-4dd2-a9dd-30b47fc83688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:59.617630 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b910eee-a6a6-4dd2-a9dd-30b47fc83688 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:59.694860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1087/2188] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:14:59 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:59.699156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce35fc6-0990-487e-9ef1-00a77ce6ff3e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:59.710068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce35fc6-0990-487e-9ef1-00a77ce6ff3e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:59.766678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ece54-a038-491f-b773-df4a6c1790d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:59.767228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ece54-a038-491f-b773-df4a6c1790d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:59.767517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ece54-a038-491f-b773-df4a6c1790d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:59.767798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ece54-a038-491f-b773-df4a6c1790d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:59.768503 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ece54-a038-491f-b773-df4a6c1790d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:59.770546 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ece54-a038-491f-b773-df4a6c1790d1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:59.896765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1102/2189] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:14:59 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:14:59.900854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b910eee-a6a6-4dd2-a9dd-30b47fc83688 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:59.911836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b910eee-a6a6-4dd2-a9dd-30b47fc83688 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:59.916385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3f9f8-6f64-4921-a497-85a5c85a6775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:59.921209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3f9f8-6f64-4921-a497-85a5c85a6775 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:59.921209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3f9f8-6f64-4921-a497-85a5c85a6775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:59.921209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3f9f8-6f64-4921-a497-85a5c85a6775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:59.921209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3f9f8-6f64-4921-a497-85a5c85a6775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:59.921209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3f9f8-6f64-4921-a497-85a5c85a6775 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:14:59.931325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1088/2190] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:14:59.935522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ece54-a038-491f-b773-df4a6c1790d1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:14:59.945758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ece54-a038-491f-b773-df4a6c1790d1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:14:59.950439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15683bfb-b5b2-44d1-adda-e4532d603708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:14:59.951092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15683bfb-b5b2-44d1-adda-e4532d603708 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:14:59.951377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15683bfb-b5b2-44d1-adda-e4532d603708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:14:59.951655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15683bfb-b5b2-44d1-adda-e4532d603708 admin admin] PATH_INFO: `/v3/projects/72a3e387c6064e06847dcd92b46be4c3/users/ce3e50180d624207bd7d59b766236ec7/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:14:59.967283 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15683bfb-b5b2-44d1-adda-e4532d603708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a3e387c6064e06847dcd92b46be4c3, user_id=ce3e50180d624207bd7d59b766236ec7, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:14:59.969968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15683bfb-b5b2-44d1-adda-e4532d603708 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:00.061795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:00.067412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1103/2191] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:14:59 2022] PUT /identity/v3/projects/72a3e387c6064e06847dcd92b46be4c3/users/ce3e50180d624207bd7d59b766236ec7/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:00.071526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3f9f8-6f64-4921-a497-85a5c85a6775 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:00.082422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3f9f8-6f64-4921-a497-85a5c85a6775 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:00.141375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9feba8d-6033-4c8f-b9df-351770b28b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:00.142003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9feba8d-6033-4c8f-b9df-351770b28b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:00.142289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9feba8d-6033-4c8f-b9df-351770b28b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:00.142567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9feba8d-6033-4c8f-b9df-351770b28b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:00.143274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9feba8d-6033-4c8f-b9df-351770b28b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:00.144908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9feba8d-6033-4c8f-b9df-351770b28b17 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:00.156733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1089/2192] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:00.160778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15683bfb-b5b2-44d1-adda-e4532d603708 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:00.172541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15683bfb-b5b2-44d1-adda-e4532d603708 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:00.177253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fddb4e-52b3-440f-894f-334593a58dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:00.177934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fddb4e-52b3-440f-894f-334593a58dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:00.178219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fddb4e-52b3-440f-894f-334593a58dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:00.178498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fddb4e-52b3-440f-894f-334593a58dc6 admin admin] PATH_INFO: `/v3/projects/72a3e387c6064e06847dcd92b46be4c3/users/ce3e50180d624207bd7d59b766236ec7/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:00.180463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fddb4e-52b3-440f-894f-334593a58dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a3e387c6064e06847dcd92b46be4c3, user_id=ce3e50180d624207bd7d59b766236ec7, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:00.183190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fddb4e-52b3-440f-894f-334593a58dc6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:00.195946 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1104/2193] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:00 2022] PUT /identity/v3/projects/72a3e387c6064e06847dcd92b46be4c3/users/ce3e50180d624207bd7d59b766236ec7/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:01.546937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094d4e3-692f-46c8-b321-8f458a5e9eea None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:01.547418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094d4e3-692f-46c8-b321-8f458a5e9eea None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:01.547513 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094d4e3-692f-46c8-b321-8f458a5e9eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:01.575940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e094d4e3-692f-46c8-b321-8f458a5e9eea None None] MFA Rules not processed for user `ce3e50180d624207bd7d59b766236ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:01.577247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e094d4e3-692f-46c8-b321-8f458a5e9eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:01.585934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:01.664762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1090/2194] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:01.676878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fddb4e-52b3-440f-894f-334593a58dc6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:01.678639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-66fddb4e-52b3-440f-894f-334593a58dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:01.735291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fddb4e-52b3-440f-894f-334593a58dc6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:01.738891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97b27a-5bf4-4249-b6ee-42bfb7dd191a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:01.739500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97b27a-5bf4-4249-b6ee-42bfb7dd191a service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:01.739784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97b27a-5bf4-4249-b6ee-42bfb7dd191a service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:01.740056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97b27a-5bf4-4249-b6ee-42bfb7dd191a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:01.796495 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97b27a-5bf4-4249-b6ee-42bfb7dd191a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:01.797961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97b27a-5bf4-4249-b6ee-42bfb7dd191a service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:01.807703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1105/2195] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:03.621255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094d4e3-692f-46c8-b321-8f458a5e9eea None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:03.631710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094d4e3-692f-46c8-b321-8f458a5e9eea None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:03.635577 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7be76-ea86-442f-9671-be6c07bf674b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:03.636199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7be76-ea86-442f-9671-be6c07bf674b service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:03.636507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7be76-ea86-442f-9671-be6c07bf674b service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:03.636783 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7be76-ea86-442f-9671-be6c07bf674b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:03.638580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-19e7be76-ea86-442f-9671-be6c07bf674b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:03.704178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7be76-ea86-442f-9671-be6c07bf674b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:03.705687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7be76-ea86-442f-9671-be6c07bf674b service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:03.719518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1091/2196] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:09.791629 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97b27a-5bf4-4249-b6ee-42bfb7dd191a service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:09.800805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97b27a-5bf4-4249-b6ee-42bfb7dd191a service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:09.804520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bf6d4-d4e6-4e23-83ec-0d892124b3f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:09.805166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bf6d4-d4e6-4e23-83ec-0d892124b3f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:09.805451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bf6d4-d4e6-4e23-83ec-0d892124b3f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:09.805730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bf6d4-d4e6-4e23-83ec-0d892124b3f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:09.815578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bf6d4-d4e6-4e23-83ec-0d892124b3f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:09.817123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bf6d4-d4e6-4e23-83ec-0d892124b3f9 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:09.826895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1106/2197] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:13.713535 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=55 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:15.345022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec3aa5-efef-46b4-b557-9979e6524bb9 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:15.345714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec3aa5-efef-46b4-b557-9979e6524bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:15.345714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec3aa5-efef-46b4-b557-9979e6524bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:15.366676 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-13ec3aa5-efef-46b4-b557-9979e6524bb9 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:15.368119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-13ec3aa5-efef-46b4-b557-9979e6524bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:15.439478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1092/2198] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:15:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:15.487536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bf6d4-d4e6-4e23-83ec-0d892124b3f9 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:15.488948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a19bf6d4-d4e6-4e23-83ec-0d892124b3f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:15.562257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bf6d4-d4e6-4e23-83ec-0d892124b3f9 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:15.567916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25844b18-3941-4fd9-9e28-9e019b2a9348 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:15.568531 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25844b18-3941-4fd9-9e28-9e019b2a9348 service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:15.568812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25844b18-3941-4fd9-9e28-9e019b2a9348 service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:15.569120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25844b18-3941-4fd9-9e28-9e019b2a9348 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:15.577400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25844b18-3941-4fd9-9e28-9e019b2a9348 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:15.578962 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25844b18-3941-4fd9-9e28-9e019b2a9348 service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:15.587760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1107/2199] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:15:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:15:25.433168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=25 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:25.584488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:32.923087 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec3aa5-efef-46b4-b557-9979e6524bb9 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:32.926012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-13ec3aa5-efef-46b4-b557-9979e6524bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:33.027133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec3aa5-efef-46b4-b557-9979e6524bb9 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:33.036421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe141a7-d2fb-4a4d-9b1f-33803c96b940 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:33.037408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe141a7-d2fb-4a4d-9b1f-33803c96b940 service placement] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:33.037690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe141a7-d2fb-4a4d-9b1f-33803c96b940 service placement] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:33.038114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe141a7-d2fb-4a4d-9b1f-33803c96b940 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:33.040703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-afe141a7-d2fb-4a4d-9b1f-33803c96b940 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:33.057237 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe141a7-d2fb-4a4d-9b1f-33803c96b940 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:33.059615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe141a7-d2fb-4a4d-9b1f-33803c96b940 service placement] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:33.075416 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1093/2200] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:43.068277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=32 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:44.004819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54833487-8c4c-4ae5-b912-feb5b68ae2d3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:44.010613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54833487-8c4c-4ae5-b912-feb5b68ae2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:44.010613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54833487-8c4c-4ae5-b912-feb5b68ae2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:44.026641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-54833487-8c4c-4ae5-b912-feb5b68ae2d3 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:44.029027 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-54833487-8c4c-4ae5-b912-feb5b68ae2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:44.037985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1108/2201] 10.222.0.86 () {58 vars in 967 bytes} [Thu Jul 28 18:15:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:44.062936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe141a7-d2fb-4a4d-9b1f-33803c96b940 service placement] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:44.063037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe141a7-d2fb-4a4d-9b1f-33803c96b940 service placement] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:44.068214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59c560-ca6a-4038-97b7-d10f34253c21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:44.068826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59c560-ca6a-4038-97b7-d10f34253c21 service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:44.069477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59c560-ca6a-4038-97b7-d10f34253c21 service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:44.069477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59c560-ca6a-4038-97b7-d10f34253c21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:44.077850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59c560-ca6a-4038-97b7-d10f34253c21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:44.079410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59c560-ca6a-4038-97b7-d10f34253c21 service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:44.088673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1094/2202] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:15:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:15:48.404322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708f880-03b8-480a-a8e8-b989bbd766ad None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:48.404819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708f880-03b8-480a-a8e8-b989bbd766ad None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:48.404903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708f880-03b8-480a-a8e8-b989bbd766ad None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:48.406285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1109/2203] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:15:48.439814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee80065-7976-4907-bc06-7c2ddb5e74e6 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:48.440319 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee80065-7976-4907-bc06-7c2ddb5e74e6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:48.440420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee80065-7976-4907-bc06-7c2ddb5e74e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:48.461190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3ee80065-7976-4907-bc06-7c2ddb5e74e6 None None] MFA Rules not processed for user `b20564a83953433290aa3fabf884370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:48.462472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3ee80065-7976-4907-bc06-7c2ddb5e74e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:48.472001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1095/2204] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:15:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:48.483404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708f880-03b8-480a-a8e8-b989bbd766ad None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:48.485104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1708f880-03b8-480a-a8e8-b989bbd766ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:48.502156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708f880-03b8-480a-a8e8-b989bbd766ad None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:48.507786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b786fe-d926-415c-b0a1-4a5e37d58c7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:48.508387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b786fe-d926-415c-b0a1-4a5e37d58c7c service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:48.508667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b786fe-d926-415c-b0a1-4a5e37d58c7c service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:48.508939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b786fe-d926-415c-b0a1-4a5e37d58c7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:48.517233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b786fe-d926-415c-b0a1-4a5e37d58c7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:48.518936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b786fe-d926-415c-b0a1-4a5e37d58c7c service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:48.528898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1110/2205] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:48.550036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee80065-7976-4907-bc06-7c2ddb5e74e6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:48.551707 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3ee80065-7976-4907-bc06-7c2ddb5e74e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:48.626098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee80065-7976-4907-bc06-7c2ddb5e74e6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:48.631425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a6d6a-8012-4b3f-b47f-6bca40ac6053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:48.632072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a6d6a-8012-4b3f-b47f-6bca40ac6053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:48.632357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a6d6a-8012-4b3f-b47f-6bca40ac6053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:48.632635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a6d6a-8012-4b3f-b47f-6bca40ac6053 admin admin] PATH_INFO: `/v3/users/a794c5d63c0f499db264113d64ae63b5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:48.633875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a6d6a-8012-4b3f-b47f-6bca40ac6053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a794c5d63c0f499db264113d64ae63b5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:48.635814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a6d6a-8012-4b3f-b47f-6bca40ac6053 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:48.844769 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d94a6d6a-8012-4b3f-b47f-6bca40ac6053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a794c5d63c0f499db264113d64ae63b5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:48.852221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d94a6d6a-8012-4b3f-b47f-6bca40ac6053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a794c5d63c0f499db264113d64ae63b5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:48.916288 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d94a6d6a-8012-4b3f-b47f-6bca40ac6053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a794c5d63c0f499db264113d64ae63b5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:48.916907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d94a6d6a-8012-4b3f-b47f-6bca40ac6053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a794c5d63c0f499db264113d64ae63b5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:48.930972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-d94a6d6a-8012-4b3f-b47f-6bca40ac6053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a794c5d63c0f499db264113d64ae63b5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:48.937337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1096/2206] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:48 2022] DELETE /identity/v3/users/a794c5d63c0f499db264113d64ae63b5 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:49.261480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b786fe-d926-415c-b0a1-4a5e37d58c7c service placement] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:49.276567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b786fe-d926-415c-b0a1-4a5e37d58c7c service placement] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:49.335846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:49.336521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:49.336805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:49.337128 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] PATH_INFO: `/v3/projects/85d8da2c21474c958de9039caa0b547d` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:49.338296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] RBAC: Authorizing `identity:delete_project(project_id=85d8da2c21474c958de9039caa0b547d)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:49.340197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:49.444644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85d8da2c21474c958de9039caa0b547d', 'request_id': 'req-03de2f66-ae57-4771-a5ac-3fe4637c596a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:49.450223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85d8da2c21474c958de9039caa0b547d', 'request_id': 'req-03de2f66-ae57-4771-a5ac-3fe4637c596a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:49.456425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85d8da2c21474c958de9039caa0b547d', 'request_id': 'req-03de2f66-ae57-4771-a5ac-3fe4637c596a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:49.533905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] The token cache is being invalidate because project 85d8da2c21474c958de9039caa0b547d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:49.534196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:49.534839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-03de2f66-ae57-4771-a5ac-3fe4637c596a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:49.536928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1111/2207] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:49 2022] DELETE /identity/v3/projects/85d8da2c21474c958de9039caa0b547d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:15:49.542028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a45186-b75c-4fd6-82ba-ceff16b86f95 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:49.542516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a45186-b75c-4fd6-82ba-ceff16b86f95 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:49.542626 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a45186-b75c-4fd6-82ba-ceff16b86f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:49.563024 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-b3a45186-b75c-4fd6-82ba-ceff16b86f95 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:49.564245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b3a45186-b75c-4fd6-82ba-ceff16b86f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:49.614605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:49.622882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1097/2208] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:49.630523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dab444-afe6-4dac-be86-bbda95500c92 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:49.632338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dab444-afe6-4dac-be86-bbda95500c92 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:49.632338 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dab444-afe6-4dac-be86-bbda95500c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:49.650542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c8dab444-afe6-4dac-be86-bbda95500c92 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:49.651752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c8dab444-afe6-4dac-be86-bbda95500c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:49.659659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1112/2209] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:49.664338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a45186-b75c-4fd6-82ba-ceff16b86f95 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:49.673958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a45186-b75c-4fd6-82ba-ceff16b86f95 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:49.679245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c6aec-7665-4964-9c35-ac4a4bbe6006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:49.679787 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c6aec-7665-4964-9c35-ac4a4bbe6006 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:49.680068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c6aec-7665-4964-9c35-ac4a4bbe6006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:49.680343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c6aec-7665-4964-9c35-ac4a4bbe6006 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:49.681243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c6aec-7665-4964-9c35-ac4a4bbe6006 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:15:49.681784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c6aec-7665-4964-9c35-ac4a4bbe6006 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:49.683728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c6aec-7665-4964-9c35-ac4a4bbe6006 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:49.701848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1098/2210] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:15:49.706206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dab444-afe6-4dac-be86-bbda95500c92 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:49.716704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dab444-afe6-4dac-be86-bbda95500c92 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:49.721961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc419f8-4776-403c-a383-8fc213cd518b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:49.722588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc419f8-4776-403c-a383-8fc213cd518b admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:49.722872 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc419f8-4776-403c-a383-8fc213cd518b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:49.723644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc419f8-4776-403c-a383-8fc213cd518b admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:49.724333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc419f8-4776-403c-a383-8fc213cd518b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:49.726287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc419f8-4776-403c-a383-8fc213cd518b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:49.829426 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1113/2211] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:15:49 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:49.834097 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c6aec-7665-4964-9c35-ac4a4bbe6006 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:49.844478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c6aec-7665-4964-9c35-ac4a4bbe6006 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:49.900837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940834d9-ba42-4064-a66a-461c16cb3698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:49.901511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940834d9-ba42-4064-a66a-461c16cb3698 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:49.906388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940834d9-ba42-4064-a66a-461c16cb3698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:49.906388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940834d9-ba42-4064-a66a-461c16cb3698 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:49.906388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940834d9-ba42-4064-a66a-461c16cb3698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:49.906388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940834d9-ba42-4064-a66a-461c16cb3698 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:49.985324 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1099/2212] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:15:49 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:49.989747 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc419f8-4776-403c-a383-8fc213cd518b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:50.000720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc419f8-4776-403c-a383-8fc213cd518b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:50.005434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758ee6-e735-4316-9e4d-68d05501ba03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:50.006052 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758ee6-e735-4316-9e4d-68d05501ba03 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:50.006370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758ee6-e735-4316-9e4d-68d05501ba03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:50.006654 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758ee6-e735-4316-9e4d-68d05501ba03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:50.007389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a758ee6-e735-4316-9e4d-68d05501ba03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:50.009039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a758ee6-e735-4316-9e4d-68d05501ba03 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:50.021083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1114/2213] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:50.025323 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940834d9-ba42-4064-a66a-461c16cb3698 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:50.036922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940834d9-ba42-4064-a66a-461c16cb3698 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:50.041942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc28734-7c55-4741-8413-aa61a696b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:50.044517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc28734-7c55-4741-8413-aa61a696b9fd admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:50.044517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc28734-7c55-4741-8413-aa61a696b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:50.044517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc28734-7c55-4741-8413-aa61a696b9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:50.044517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc28734-7c55-4741-8413-aa61a696b9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:50.045603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc28734-7c55-4741-8413-aa61a696b9fd admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:50.056898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1100/2214] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:50.062539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758ee6-e735-4316-9e4d-68d05501ba03 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:50.074256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758ee6-e735-4316-9e4d-68d05501ba03 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:50.079173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b07520-2594-4884-9e3c-b9f56756b0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:50.079838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07520-2594-4884-9e3c-b9f56756b0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:50.080117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07520-2594-4884-9e3c-b9f56756b0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:50.080392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07520-2594-4884-9e3c-b9f56756b0d7 admin admin] PATH_INFO: `/v3/projects/e207ffc0829d41be9c8f276e8c4e4ff2/users/002120fd840a45f390a4fb12b7c578d8/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:50.095956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b07520-2594-4884-9e3c-b9f56756b0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e207ffc0829d41be9c8f276e8c4e4ff2, user_id=002120fd840a45f390a4fb12b7c578d8, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:50.099251 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b07520-2594-4884-9e3c-b9f56756b0d7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:50.193275 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1115/2215] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:50 2022] PUT /identity/v3/projects/e207ffc0829d41be9c8f276e8c4e4ff2/users/002120fd840a45f390a4fb12b7c578d8/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:50.205739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc48e8-bdda-4012-8269-342cd0604ec4 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:50.206035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc48e8-bdda-4012-8269-342cd0604ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:50.206319 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc48e8-bdda-4012-8269-342cd0604ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:50.234983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-b6cc48e8-bdda-4012-8269-342cd0604ec4 None None] MFA Rules not processed for user `002120fd840a45f390a4fb12b7c578d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:50.236224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b6cc48e8-bdda-4012-8269-342cd0604ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:50.312283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1101/2216] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:50.327438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b07520-2594-4884-9e3c-b9f56756b0d7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:50.329391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d2b07520-2594-4884-9e3c-b9f56756b0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:50.340382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:50.390286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b07520-2594-4884-9e3c-b9f56756b0d7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:50.395549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf63174-c235-41ba-8309-4ba222346c42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:50.396221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf63174-c235-41ba-8309-4ba222346c42 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:50.396505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf63174-c235-41ba-8309-4ba222346c42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:50.396780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf63174-c235-41ba-8309-4ba222346c42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:50.405197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf63174-c235-41ba-8309-4ba222346c42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:50.406620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf63174-c235-41ba-8309-4ba222346c42 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:50.415541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1116/2217] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:52.067664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8f5b1-1c43-4f13-aa17-0b35864e14f5 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:52.068157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8f5b1-1c43-4f13-aa17-0b35864e14f5 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:52.068285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8f5b1-1c43-4f13-aa17-0b35864e14f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:52.087567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0ce8f5b1-1c43-4f13-aa17-0b35864e14f5 None None] MFA Rules not processed for user `5ef698c4f57f44d387550921a288d7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:52.088772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0ce8f5b1-1c43-4f13-aa17-0b35864e14f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:52.139487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:52.146976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1102/2218] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:54.391149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf63174-c235-41ba-8309-4ba222346c42 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:54.393973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1bf63174-c235-41ba-8309-4ba222346c42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:54.466921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf63174-c235-41ba-8309-4ba222346c42 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:54.472025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c80dec-37ab-493f-aeda-ca9d3be5d87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:54.472819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c80dec-37ab-493f-aeda-ca9d3be5d87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:54.473686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c80dec-37ab-493f-aeda-ca9d3be5d87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:54.473980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c80dec-37ab-493f-aeda-ca9d3be5d87c admin admin] PATH_INFO: `/v3/users/7253c33029f24d048490fb1d3083f879` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:54.475238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c80dec-37ab-493f-aeda-ca9d3be5d87c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7253c33029f24d048490fb1d3083f879)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:54.477250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c80dec-37ab-493f-aeda-ca9d3be5d87c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:54.537745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8f5b1-1c43-4f13-aa17-0b35864e14f5 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:54.550211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8f5b1-1c43-4f13-aa17-0b35864e14f5 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:54.554768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:54.555403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:54.555689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:54.555965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] PATH_INFO: `/v3/users/002120fd840a45f390a4fb12b7c578d8` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:54.557200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002120fd840a45f390a4fb12b7c578d8)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:54.559118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:54.686644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e9c80dec-37ab-493f-aeda-ca9d3be5d87c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7253c33029f24d048490fb1d3083f879'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:54.687298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e9c80dec-37ab-493f-aeda-ca9d3be5d87c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7253c33029f24d048490fb1d3083f879'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:54.694972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e9c80dec-37ab-493f-aeda-ca9d3be5d87c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7253c33029f24d048490fb1d3083f879'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:54.753243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002120fd840a45f390a4fb12b7c578d8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:54.761461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002120fd840a45f390a4fb12b7c578d8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:54.775198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e9c80dec-37ab-493f-aeda-ca9d3be5d87c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7253c33029f24d048490fb1d3083f879'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:54.775825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e9c80dec-37ab-493f-aeda-ca9d3be5d87c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7253c33029f24d048490fb1d3083f879'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:54.797156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1117/2219] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:54 2022] DELETE /identity/v3/users/7253c33029f24d048490fb1d3083f879 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:54.801965 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63d5db-12ff-42ee-8b12-5438b1464b89 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:54.802248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63d5db-12ff-42ee-8b12-5438b1464b89 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:54.802525 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63d5db-12ff-42ee-8b12-5438b1464b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:54.824480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3f63d5db-12ff-42ee-8b12-5438b1464b89 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:54.825896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3f63d5db-12ff-42ee-8b12-5438b1464b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:54.851603 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:54.871988 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002120fd840a45f390a4fb12b7c578d8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:54.872715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002120fd840a45f390a4fb12b7c578d8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:54.886547 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002120fd840a45f390a4fb12b7c578d8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:54.889385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1118/2220] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:15:54.896816 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1103/2221] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:54 2022] DELETE /identity/v3/users/002120fd840a45f390a4fb12b7c578d8 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:15:54.898138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44e918-100c-4eac-a698-d735a7d83f22 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:54.898430 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44e918-100c-4eac-a698-d735a7d83f22 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:54.898819 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44e918-100c-4eac-a698-d735a7d83f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:54.901158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:54.902530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:54.918927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4c44e918-100c-4eac-a698-d735a7d83f22 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:54.920472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4c44e918-100c-4eac-a698-d735a7d83f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:54.995501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1119/2222] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:15:54.996026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa881e-93a4-4592-87f4-16fe1218db75 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:55.001142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07776f-9ac1-4509-8895-705f1229fd01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:55.001786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07776f-9ac1-4509-8895-705f1229fd01 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:55.002134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07776f-9ac1-4509-8895-705f1229fd01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:55.002410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07776f-9ac1-4509-8895-705f1229fd01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:55.012264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44e918-100c-4eac-a698-d735a7d83f22 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:55.017993 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07776f-9ac1-4509-8895-705f1229fd01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:55.019598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07776f-9ac1-4509-8895-705f1229fd01 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:55.031526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1104/2223] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:15:55.031859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44e918-100c-4eac-a698-d735a7d83f22 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:55.035527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da9c7d-f9ca-4563-b271-67e0ed3967e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:55.036187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da9c7d-f9ca-4563-b271-67e0ed3967e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:55.036490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da9c7d-f9ca-4563-b271-67e0ed3967e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:55.036763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da9c7d-f9ca-4563-b271-67e0ed3967e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:55.051458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da9c7d-f9ca-4563-b271-67e0ed3967e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:55.052989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da9c7d-f9ca-4563-b271-67e0ed3967e0 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:55.064859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1120/2224] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:15:55.571841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07776f-9ac1-4509-8895-705f1229fd01 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:55.573395 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cb07776f-9ac1-4509-8895-705f1229fd01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:55.583947 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da9c7d-f9ca-4563-b271-67e0ed3967e0 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:55.585478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-50da9c7d-f9ca-4563-b271-67e0ed3967e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:55.587497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07776f-9ac1-4509-8895-705f1229fd01 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:55.594230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:55.595406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:55.595406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:55.595608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] PATH_INFO: `/v3/projects/e207ffc0829d41be9c8f276e8c4e4ff2` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:55.596930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e207ffc0829d41be9c8f276e8c4e4ff2)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:55.599616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:55.601883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da9c7d-f9ca-4563-b271-67e0ed3967e0 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:55.613513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:55.614839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:55.615110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:55.615389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] PATH_INFO: `/v3/projects/d86f5d967df54762aefd8b5311f45dda` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:55.616691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d86f5d967df54762aefd8b5311f45dda)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:55.618880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:55.770881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd86f5d967df54762aefd8b5311f45dda', 'request_id': 'req-2f7933be-073b-4eb5-bce8-a148be4485dc'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:55.771290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e207ffc0829d41be9c8f276e8c4e4ff2', 'request_id': 'req-ea50c143-f1cf-414c-84c1-c73ab8b55541'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:55.777718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e207ffc0829d41be9c8f276e8c4e4ff2', 'request_id': 'req-ea50c143-f1cf-414c-84c1-c73ab8b55541'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:55.777909 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd86f5d967df54762aefd8b5311f45dda', 'request_id': 'req-2f7933be-073b-4eb5-bce8-a148be4485dc'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:55.784826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd86f5d967df54762aefd8b5311f45dda', 'request_id': 'req-2f7933be-073b-4eb5-bce8-a148be4485dc'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:55.849861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e207ffc0829d41be9c8f276e8c4e4ff2', 'request_id': 'req-ea50c143-f1cf-414c-84c1-c73ab8b55541'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:55.857584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:55.862189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] The token cache is being invalidate because project e207ffc0829d41be9c8f276e8c4e4ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:55.862486 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:55.863131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ea50c143-f1cf-414c-84c1-c73ab8b55541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:55.864705 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1105/2225] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:55 2022] DELETE /identity/v3/projects/e207ffc0829d41be9c8f276e8c4e4ff2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:55.869795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c910cad-e35b-402b-8303-c4330629583e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:55.870084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c910cad-e35b-402b-8303-c4330629583e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:55.870363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c910cad-e35b-402b-8303-c4330629583e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:55.889712 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0c910cad-e35b-402b-8303-c4330629583e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:55.890931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0c910cad-e35b-402b-8303-c4330629583e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:55.946027 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] The token cache is being invalidate because project d86f5d967df54762aefd8b5311f45dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:55.946560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:55.947194 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:55.948742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1121/2226] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:55 2022] DELETE /identity/v3/projects/d86f5d967df54762aefd8b5311f45dda => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:15:55.951385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1106/2227] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:15:55.958326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:55.958326 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:55.959925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973d6f7-8332-4bd3-bcbf-61d4564f1976 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:55.960217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973d6f7-8332-4bd3-bcbf-61d4564f1976 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:55.960494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973d6f7-8332-4bd3-bcbf-61d4564f1976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:55.980215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e973d6f7-8332-4bd3-bcbf-61d4564f1976 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:55.981468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e973d6f7-8332-4bd3-bcbf-61d4564f1976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:55.990685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1107/2228] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:55.996399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973d6f7-8332-4bd3-bcbf-61d4564f1976 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:55.999341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:56.005950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973d6f7-8332-4bd3-bcbf-61d4564f1976 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:56.010810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ae2c8-c8b2-484f-b370-94a8b9caaf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:56.011431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ae2c8-c8b2-484f-b370-94a8b9caaf77 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:56.011771 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ae2c8-c8b2-484f-b370-94a8b9caaf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:56.012192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ae2c8-c8b2-484f-b370-94a8b9caaf77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:56.013132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ae2c8-c8b2-484f-b370-94a8b9caaf77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:15:56.013446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ae2c8-c8b2-484f-b370-94a8b9caaf77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:56.015044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ae2c8-c8b2-484f-b370-94a8b9caaf77 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:56.020685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7933be-073b-4eb5-bce8-a148be4485dc admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:56.027054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:56.031098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1108/2229] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:15:56.031253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:56.031253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:56.031253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] PATH_INFO: `/v3/users/8184af003d7c4bcb82d4685e3ef66941` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:56.031253 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8184af003d7c4bcb82d4685e3ef66941)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:56.032062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:56.032140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ae2c8-c8b2-484f-b370-94a8b9caaf77 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:56.043062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ae2c8-c8b2-484f-b370-94a8b9caaf77 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:56.048060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab6c3b-6677-4914-80f1-732e43c6f9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:56.048685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab6c3b-6677-4914-80f1-732e43c6f9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:56.048994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab6c3b-6677-4914-80f1-732e43c6f9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:56.049289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab6c3b-6677-4914-80f1-732e43c6f9d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:56.050131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab6c3b-6677-4914-80f1-732e43c6f9d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:56.052081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab6c3b-6677-4914-80f1-732e43c6f9d7 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:56.195044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1109/2230] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:15:56 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:15:56.199137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab6c3b-6677-4914-80f1-732e43c6f9d7 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:56.266305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab6c3b-6677-4914-80f1-732e43c6f9d7 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:56.271832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661e48a-7bf4-4843-a424-074e11d11906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:56.272458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661e48a-7bf4-4843-a424-074e11d11906 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:56.272783 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661e48a-7bf4-4843-a424-074e11d11906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:56.273244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661e48a-7bf4-4843-a424-074e11d11906 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:56.273770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d661e48a-7bf4-4843-a424-074e11d11906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:56.275721 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d661e48a-7bf4-4843-a424-074e11d11906 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:56.302606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8184af003d7c4bcb82d4685e3ef66941'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:56.303271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8184af003d7c4bcb82d4685e3ef66941'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:56.311361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8184af003d7c4bcb82d4685e3ef66941'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:56.359843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1110/2231] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:15:56 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:15:56.363518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661e48a-7bf4-4843-a424-074e11d11906 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:56.378201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661e48a-7bf4-4843-a424-074e11d11906 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:56.383894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a44e5-9312-401e-81b6-2a4886ebff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:56.384510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a44e5-9312-401e-81b6-2a4886ebff97 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:56.384832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a44e5-9312-401e-81b6-2a4886ebff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:56.385146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a44e5-9312-401e-81b6-2a4886ebff97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:56.385966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a44e5-9312-401e-81b6-2a4886ebff97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:56.387580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a44e5-9312-401e-81b6-2a4886ebff97 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:56.392186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8184af003d7c4bcb82d4685e3ef66941'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:56.393594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8184af003d7c4bcb82d4685e3ef66941'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:56.400327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1111/2232] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:15:56.404522 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a44e5-9312-401e-81b6-2a4886ebff97 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:56.406269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f95a44e5-9312-401e-81b6-2a4886ebff97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:56.416413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1122/2233] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:55 2022] DELETE /identity/v3/users/8184af003d7c4bcb82d4685e3ef66941 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:56.422711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a44e5-9312-401e-81b6-2a4886ebff97 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:56.486482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf4206-9699-4694-8211-224cb455de84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:56.486946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf4206-9699-4694-8211-224cb455de84 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:56.487233 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf4206-9699-4694-8211-224cb455de84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:56.487515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf4206-9699-4694-8211-224cb455de84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:56.488224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf4206-9699-4694-8211-224cb455de84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:56.489978 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf4206-9699-4694-8211-224cb455de84 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:56.500292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1112/2234] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:15:56.504881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:56.515496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c11663-5476-4d5f-82c2-4c7ddbfa33d7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:56.521072 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12daab-d2e0-4580-9c19-af88314a2e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:56.521900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12daab-d2e0-4580-9c19-af88314a2e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:56.522275 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12daab-d2e0-4580-9c19-af88314a2e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:56.522556 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12daab-d2e0-4580-9c19-af88314a2e65 admin admin] PATH_INFO: `/v3/projects/10251a1b077c4066a1898bdab6bb37dd/users/7106f76d179e4775b0a8e7e3b819df9d/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:56.538061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12daab-d2e0-4580-9c19-af88314a2e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10251a1b077c4066a1898bdab6bb37dd, user_id=7106f76d179e4775b0a8e7e3b819df9d, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:56.540730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12daab-d2e0-4580-9c19-af88314a2e65 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:56.590389 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:56.656686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1123/2235] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:56 2022] PUT /identity/v3/projects/10251a1b077c4066a1898bdab6bb37dd/users/7106f76d179e4775b0a8e7e3b819df9d/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:56.669936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262d00d-ce7a-4fa2-a8d8-b24dd74b1596 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:56.671975 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262d00d-ce7a-4fa2-a8d8-b24dd74b1596 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:56.672283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262d00d-ce7a-4fa2-a8d8-b24dd74b1596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:56.704315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5262d00d-ce7a-4fa2-a8d8-b24dd74b1596 None None] MFA Rules not processed for user `7106f76d179e4775b0a8e7e3b819df9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:56.706352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5262d00d-ce7a-4fa2-a8d8-b24dd74b1596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:56.803086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1113/2236] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:15:56.860604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12daab-d2e0-4580-9c19-af88314a2e65 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:56.862449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8a12daab-d2e0-4580-9c19-af88314a2e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:56.869198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262d00d-ce7a-4fa2-a8d8-b24dd74b1596 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:56.870595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5262d00d-ce7a-4fa2-a8d8-b24dd74b1596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:56.948874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262d00d-ce7a-4fa2-a8d8-b24dd74b1596 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:56.951039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12daab-d2e0-4580-9c19-af88314a2e65 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:56.954996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7edd63-b0d6-4ddd-abec-cb88b8ef0118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:56.955633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7edd63-b0d6-4ddd-abec-cb88b8ef0118 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:56.955917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7edd63-b0d6-4ddd-abec-cb88b8ef0118 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:56.961610 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7edd63-b0d6-4ddd-abec-cb88b8ef0118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:56.962047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:56.962388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:56.962564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:56.962900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] PATH_INFO: `/v3/projects/4f3fb6f4f2294a68be22107af07a5e9a` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:56.964172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3fb6f4f2294a68be22107af07a5e9a)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:56.966957 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:56.971612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7edd63-b0d6-4ddd-abec-cb88b8ef0118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:56.973826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7edd63-b0d6-4ddd-abec-cb88b8ef0118 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:56.984000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1124/2237] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:57.090504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3fb6f4f2294a68be22107af07a5e9a', 'request_id': 'req-e6415d10-1d31-4894-8148-a50a81bb420d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:57.100027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3fb6f4f2294a68be22107af07a5e9a', 'request_id': 'req-e6415d10-1d31-4894-8148-a50a81bb420d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:57.112176 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:57.216455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3fb6f4f2294a68be22107af07a5e9a', 'request_id': 'req-e6415d10-1d31-4894-8148-a50a81bb420d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:57.232673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] The token cache is being invalidate because project 4f3fb6f4f2294a68be22107af07a5e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:57.233322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:57.248772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e6415d10-1d31-4894-8148-a50a81bb420d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:57.248772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1114/2238] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:56 2022] DELETE /identity/v3/projects/4f3fb6f4f2294a68be22107af07a5e9a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:15:57.252930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c2f54-8f16-4e9f-9a3a-86c57a7dd939 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:57.254074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c2f54-8f16-4e9f-9a3a-86c57a7dd939 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:57.254603 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c2f54-8f16-4e9f-9a3a-86c57a7dd939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:57.286717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ec7c2f54-8f16-4e9f-9a3a-86c57a7dd939 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:57.289898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ec7c2f54-8f16-4e9f-9a3a-86c57a7dd939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:57.358007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:57.396232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1125/2239] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:57.406301 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fcf99-c217-42a1-ad42-a1ff5f5318c9 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:57.406591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fcf99-c217-42a1-ad42-a1ff5f5318c9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:57.406874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fcf99-c217-42a1-ad42-a1ff5f5318c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:57.459859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a17fcf99-c217-42a1-ad42-a1ff5f5318c9 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:57.459859 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a17fcf99-c217-42a1-ad42-a1ff5f5318c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:57.467320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1115/2240] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:57.472194 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c2f54-8f16-4e9f-9a3a-86c57a7dd939 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:57.482824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c2f54-8f16-4e9f-9a3a-86c57a7dd939 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:57.487646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb60c5e-911c-4cb3-b8fd-89612902ac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:57.488342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb60c5e-911c-4cb3-b8fd-89612902ac21 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:57.489174 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb60c5e-911c-4cb3-b8fd-89612902ac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:57.489459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb60c5e-911c-4cb3-b8fd-89612902ac21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:57.490172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb60c5e-911c-4cb3-b8fd-89612902ac21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:15:57.490483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb60c5e-911c-4cb3-b8fd-89612902ac21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:57.493050 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb60c5e-911c-4cb3-b8fd-89612902ac21 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:57.510401 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1126/2241] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:15:57.511797 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fcf99-c217-42a1-ad42-a1ff5f5318c9 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:57.521332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fcf99-c217-42a1-ad42-a1ff5f5318c9 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:57.526616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e227-f675-4aa1-9116-8180623d6dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:57.527261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e227-f675-4aa1-9116-8180623d6dce admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:57.527548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e227-f675-4aa1-9116-8180623d6dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:57.527962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e227-f675-4aa1-9116-8180623d6dce admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:57.534331 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e227-f675-4aa1-9116-8180623d6dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:57.534331 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e227-f675-4aa1-9116-8180623d6dce admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:57.698440 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1116/2242] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:15:57 2022] POST /identity/v3/projects => generated 381 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:57.702086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb60c5e-911c-4cb3-b8fd-89612902ac21 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:57.718384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb60c5e-911c-4cb3-b8fd-89612902ac21 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:57.793007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2754cb-7b86-4a07-8408-acd7d0e8d8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:57.793007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2754cb-7b86-4a07-8408-acd7d0e8d8b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:57.793007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2754cb-7b86-4a07-8408-acd7d0e8d8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:57.793007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2754cb-7b86-4a07-8408-acd7d0e8d8b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:57.793562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2754cb-7b86-4a07-8408-acd7d0e8d8b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:57.794260 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2754cb-7b86-4a07-8408-acd7d0e8d8b5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:57.904784 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1127/2243] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:15:57 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:57.906074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e227-f675-4aa1-9116-8180623d6dce admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:57.918410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e227-f675-4aa1-9116-8180623d6dce admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:57.923303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d2f89-b6a2-4fbf-8b7e-61faba825534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:57.923941 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d2f89-b6a2-4fbf-8b7e-61faba825534 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:57.924203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d2f89-b6a2-4fbf-8b7e-61faba825534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:57.924477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d2f89-b6a2-4fbf-8b7e-61faba825534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:57.925228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d2f89-b6a2-4fbf-8b7e-61faba825534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:57.927441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d2f89-b6a2-4fbf-8b7e-61faba825534 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:57.939895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1117/2244] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:57.943981 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2754cb-7b86-4a07-8408-acd7d0e8d8b5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:57.954788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2754cb-7b86-4a07-8408-acd7d0e8d8b5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:57.960083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c11473-ac84-476b-acd9-aa5f4640d2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:57.960722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c11473-ac84-476b-acd9-aa5f4640d2ce admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:57.961042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c11473-ac84-476b-acd9-aa5f4640d2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:57.961321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c11473-ac84-476b-acd9-aa5f4640d2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:57.962225 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c11473-ac84-476b-acd9-aa5f4640d2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:57.963996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c11473-ac84-476b-acd9-aa5f4640d2ce admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:57.976745 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1128/2245] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:57.981777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d2f89-b6a2-4fbf-8b7e-61faba825534 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:57.993550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d2f89-b6a2-4fbf-8b7e-61faba825534 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:57.999290 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e39051-25ac-4f18-9d5c-a60fbdaf37f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:57.999975 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39051-25ac-4f18-9d5c-a60fbdaf37f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:58.000480 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39051-25ac-4f18-9d5c-a60fbdaf37f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:58.000916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39051-25ac-4f18-9d5c-a60fbdaf37f3 admin admin] PATH_INFO: `/v3/projects/1b309c8941ae48b4864a368001da0679/users/9c0acd2da8d240f4a6d41d1fc0ba8816/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:58.019810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e39051-25ac-4f18-9d5c-a60fbdaf37f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b309c8941ae48b4864a368001da0679, user_id=9c0acd2da8d240f4a6d41d1fc0ba8816, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:58.022476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e39051-25ac-4f18-9d5c-a60fbdaf37f3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:58.119096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1118/2246] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:57 2022] PUT /identity/v3/projects/1b309c8941ae48b4864a368001da0679/users/9c0acd2da8d240f4a6d41d1fc0ba8816/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:58.129661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e2a20-e0ec-4309-a9e7-e13ec49e87c1 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:58.129950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e2a20-e0ec-4309-a9e7-e13ec49e87c1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:58.130212 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e2a20-e0ec-4309-a9e7-e13ec49e87c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:58.168203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c55e2a20-e0ec-4309-a9e7-e13ec49e87c1 None None] MFA Rules not processed for user `9c0acd2da8d240f4a6d41d1fc0ba8816`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:58.177729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c55e2a20-e0ec-4309-a9e7-e13ec49e87c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:58.233318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e39051-25ac-4f18-9d5c-a60fbdaf37f3 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:58.235250 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-79e39051-25ac-4f18-9d5c-a60fbdaf37f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:58.284385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1129/2247] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:58.289494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e2a20-e0ec-4309-a9e7-e13ec49e87c1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:58.327315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e39051-25ac-4f18-9d5c-a60fbdaf37f3 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:58.332039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:58.334998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6c8cb-7c52-4ba8-b762-c5288fe88095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:58.335793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6c8cb-7c52-4ba8-b762-c5288fe88095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:58.336152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6c8cb-7c52-4ba8-b762-c5288fe88095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:58.336525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6c8cb-7c52-4ba8-b762-c5288fe88095 admin admin] PATH_INFO: `/v3/users/1720f17c471e48d5a35bc2038129009f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:58.338780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6c8cb-7c52-4ba8-b762-c5288fe88095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1720f17c471e48d5a35bc2038129009f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:58.342203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6c8cb-7c52-4ba8-b762-c5288fe88095 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:58.384716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e2a20-e0ec-4309-a9e7-e13ec49e87c1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:58.389811 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc10cc2-d4fe-4dbb-854c-914366e8c686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:58.390438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc10cc2-d4fe-4dbb-854c-914366e8c686 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:58.390719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc10cc2-d4fe-4dbb-854c-914366e8c686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:58.390996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc10cc2-d4fe-4dbb-854c-914366e8c686 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:58.391675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc10cc2-d4fe-4dbb-854c-914366e8c686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:58.393740 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc10cc2-d4fe-4dbb-854c-914366e8c686 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:58.481462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1130/2248] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:15:58 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:15:58.482640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc10cc2-d4fe-4dbb-854c-914366e8c686 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:58.544312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc10cc2-d4fe-4dbb-854c-914366e8c686 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:58.552063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04084558-8c8e-46c2-9808-6c381d53824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:58.553244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04084558-8c8e-46c2-9808-6c381d53824f admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:58.554062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04084558-8c8e-46c2-9808-6c381d53824f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:58.554521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04084558-8c8e-46c2-9808-6c381d53824f admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:58.555516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04084558-8c8e-46c2-9808-6c381d53824f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:58.558477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04084558-8c8e-46c2-9808-6c381d53824f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:58.561360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-67d6c8cb-7c52-4ba8-b762-c5288fe88095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1720f17c471e48d5a35bc2038129009f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:58.569494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-67d6c8cb-7c52-4ba8-b762-c5288fe88095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1720f17c471e48d5a35bc2038129009f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:58.620409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-67d6c8cb-7c52-4ba8-b762-c5288fe88095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1720f17c471e48d5a35bc2038129009f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:58.621089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-67d6c8cb-7c52-4ba8-b762-c5288fe88095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1720f17c471e48d5a35bc2038129009f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:58.635252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-67d6c8cb-7c52-4ba8-b762-c5288fe88095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1720f17c471e48d5a35bc2038129009f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:58.642764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1131/2249] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:15:58 2022] POST /identity/v3/users => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:15:58.644100 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04084558-8c8e-46c2-9808-6c381d53824f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:58.651275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1119/2250] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:15:58 2022] DELETE /identity/v3/users/1720f17c471e48d5a35bc2038129009f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:15:58.651620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-04084558-8c8e-46c2-9808-6c381d53824f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:58.651718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499eb280-8bb7-40da-9480-92a2ef16827a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:58.651718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499eb280-8bb7-40da-9480-92a2ef16827a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:58.651718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499eb280-8bb7-40da-9480-92a2ef16827a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:58.659006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04084558-8c8e-46c2-9808-6c381d53824f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:58.662974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:58.669594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-499eb280-8bb7-40da-9480-92a2ef16827a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:58.670893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-499eb280-8bb7-40da-9480-92a2ef16827a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:58.718083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981bd40-8165-4e7e-9520-8fcf22b533ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:58.718710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981bd40-8165-4e7e-9520-8fcf22b533ac admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:58.718993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981bd40-8165-4e7e-9520-8fcf22b533ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:58.719302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981bd40-8165-4e7e-9520-8fcf22b533ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:58.720013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981bd40-8165-4e7e-9520-8fcf22b533ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:58.721696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981bd40-8165-4e7e-9520-8fcf22b533ac admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:58.735089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1132/2251] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:15:58.735284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1120/2252] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:58.736474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981bd40-8165-4e7e-9520-8fcf22b533ac admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:58.744328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499eb280-8bb7-40da-9480-92a2ef16827a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:58.745825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-499eb280-8bb7-40da-9480-92a2ef16827a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:58.746578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981bd40-8165-4e7e-9520-8fcf22b533ac admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:58.751504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fa8bb-8731-4591-99f6-d3ae4430f2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:58.752595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fa8bb-8731-4591-99f6-d3ae4430f2ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:58.752595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fa8bb-8731-4591-99f6-d3ae4430f2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:58.752820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fa8bb-8731-4591-99f6-d3ae4430f2ef admin admin] PATH_INFO: `/v3/projects/3a1f5e633db8492e9f6f3ec06926b6cc/users/1dd93f55bafe499bb07e309a95fefb4f/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:58.771782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fa8bb-8731-4591-99f6-d3ae4430f2ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1f5e633db8492e9f6f3ec06926b6cc, user_id=1dd93f55bafe499bb07e309a95fefb4f, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:58.774786 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fa8bb-8731-4591-99f6-d3ae4430f2ef admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:58.805747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499eb280-8bb7-40da-9480-92a2ef16827a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:58.809733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c86f9-e7fc-4922-9a67-463f4cf026f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:58.810354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c86f9-e7fc-4922-9a67-463f4cf026f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:58.810633 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c86f9-e7fc-4922-9a67-463f4cf026f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:58.810907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c86f9-e7fc-4922-9a67-463f4cf026f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:58.820936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c86f9-e7fc-4922-9a67-463f4cf026f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:58.822576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c86f9-e7fc-4922-9a67-463f4cf026f0 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:58.833793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1121/2253] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:15:58.843733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1133/2254] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:58 2022] PUT /identity/v3/projects/3a1f5e633db8492e9f6f3ec06926b6cc/users/1dd93f55bafe499bb07e309a95fefb4f/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:58.854986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c86f9-e7fc-4922-9a67-463f4cf026f0 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:58.865735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c86f9-e7fc-4922-9a67-463f4cf026f0 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:58.935040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4ab84-7a67-4d0b-9d0a-f98dcdcf5d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:58.935684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4ab84-7a67-4d0b-9d0a-f98dcdcf5d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:58.940929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4ab84-7a67-4d0b-9d0a-f98dcdcf5d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:58.940929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4ab84-7a67-4d0b-9d0a-f98dcdcf5d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:58.940929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4ab84-7a67-4d0b-9d0a-f98dcdcf5d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:58.940929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4ab84-7a67-4d0b-9d0a-f98dcdcf5d85 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:58.950729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1122/2255] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:15:58.954999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fa8bb-8731-4591-99f6-d3ae4430f2ef admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:58.965991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fa8bb-8731-4591-99f6-d3ae4430f2ef admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:58.972089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c911c-d7c8-4eb4-8e4e-6dd5bab149af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:58.972822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c911c-d7c8-4eb4-8e4e-6dd5bab149af admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:58.973274 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c911c-d7c8-4eb4-8e4e-6dd5bab149af admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:58.973663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c911c-d7c8-4eb4-8e4e-6dd5bab149af admin admin] PATH_INFO: `/v3/projects/3a1f5e633db8492e9f6f3ec06926b6cc/users/1dd93f55bafe499bb07e309a95fefb4f/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:58.976063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c911c-d7c8-4eb4-8e4e-6dd5bab149af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1f5e633db8492e9f6f3ec06926b6cc, user_id=1dd93f55bafe499bb07e309a95fefb4f, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:58.979099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c911c-d7c8-4eb4-8e4e-6dd5bab149af admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:58.991192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1134/2256] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:15:58 2022] PUT /identity/v3/projects/3a1f5e633db8492e9f6f3ec06926b6cc/users/1dd93f55bafe499bb07e309a95fefb4f/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:59.002115 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0e9bb-389f-4c6e-ad28-6056fa129ae7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:59.002405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0e9bb-389f-4c6e-ad28-6056fa129ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:59.002681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0e9bb-389f-4c6e-ad28-6056fa129ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:59.037511 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-4cb0e9bb-389f-4c6e-ad28-6056fa129ae7 None None] MFA Rules not processed for user `1dd93f55bafe499bb07e309a95fefb4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:59.038981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4cb0e9bb-389f-4c6e-ad28-6056fa129ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:59.131567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1123/2257] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:59.142732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c911c-d7c8-4eb4-8e4e-6dd5bab149af admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:59.145008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-793c911c-d7c8-4eb4-8e4e-6dd5bab149af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:59.151251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:59.157770 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:15:59.225206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c911c-d7c8-4eb4-8e4e-6dd5bab149af admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:59.229119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef6d95-734b-41fa-81ee-33505d8b69a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:59.229750 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef6d95-734b-41fa-81ee-33505d8b69a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:59.230029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef6d95-734b-41fa-81ee-33505d8b69a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:59.230304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef6d95-734b-41fa-81ee-33505d8b69a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:59.240181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef6d95-734b-41fa-81ee-33505d8b69a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:59.241703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef6d95-734b-41fa-81ee-33505d8b69a6 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:59.251203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1135/2258] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:15:59.298519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0e9bb-389f-4c6e-ad28-6056fa129ae7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:59.300354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4cb0e9bb-389f-4c6e-ad28-6056fa129ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:59.370715 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef6d95-734b-41fa-81ee-33505d8b69a6 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:59.376017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0e9bb-389f-4c6e-ad28-6056fa129ae7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:59.379320 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef6d95-734b-41fa-81ee-33505d8b69a6 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:59.380886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:59.381588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:59.381875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:59.382153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] PATH_INFO: `/v3/projects/b05c462036494fd5840de730fdfcb7d7` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:59.383401 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cb6b1-f656-49d6-bd69-e309791a89dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:59.383555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b05c462036494fd5840de730fdfcb7d7)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:59.384207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cb6b1-f656-49d6-bd69-e309791a89dc service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:59.384613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cb6b1-f656-49d6-bd69-e309791a89dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:59.385004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cb6b1-f656-49d6-bd69-e309791a89dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:59.385498 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:59.387119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1f6cb6b1-f656-49d6-bd69-e309791a89dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:59.447876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cb6b1-f656-49d6-bd69-e309791a89dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:59.449382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cb6b1-f656-49d6-bd69-e309791a89dc service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:59.458184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1136/2259] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:15:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:15:59.508595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b05c462036494fd5840de730fdfcb7d7', 'request_id': 'req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:59.514168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b05c462036494fd5840de730fdfcb7d7', 'request_id': 'req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:59.591009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b05c462036494fd5840de730fdfcb7d7', 'request_id': 'req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:59.603528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] The token cache is being invalidate because project b05c462036494fd5840de730fdfcb7d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:15:59.603821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:59.604484 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-206d0e9c-0af2-4b2e-a870-ecbd11c4d497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:15:59.606057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1124/2260] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:15:59 2022] DELETE /identity/v3/projects/b05c462036494fd5840de730fdfcb7d7 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:15:59.610948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fc257-15b9-45e9-b942-1e5d5e8aa966 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:59.611220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fc257-15b9-45e9-b942-1e5d5e8aa966 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:59.611564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fc257-15b9-45e9-b942-1e5d5e8aa966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:59.633163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-329fc257-15b9-45e9-b942-1e5d5e8aa966 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:59.634644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-329fc257-15b9-45e9-b942-1e5d5e8aa966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:59.697341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1137/2261] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:59.705417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d78ae-15d1-4578-b617-154ade6f0085 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:59.705711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d78ae-15d1-4578-b617-154ade6f0085 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:59.705996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d78ae-15d1-4578-b617-154ade6f0085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:59.728051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5c1d78ae-15d1-4578-b617-154ade6f0085 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:15:59.729426 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5c1d78ae-15d1-4578-b617-154ade6f0085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:15:59.738308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1125/2262] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:15:59.742742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fc257-15b9-45e9-b942-1e5d5e8aa966 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:59.754053 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fc257-15b9-45e9-b942-1e5d5e8aa966 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:59.758738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e068035-ab54-4b76-934b-96d174296f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:59.759361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e068035-ab54-4b76-934b-96d174296f19 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:59.759652 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e068035-ab54-4b76-934b-96d174296f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:59.759918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e068035-ab54-4b76-934b-96d174296f19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:59.760623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e068035-ab54-4b76-934b-96d174296f19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:15:59.760932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e068035-ab54-4b76-934b-96d174296f19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:59.766125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e068035-ab54-4b76-934b-96d174296f19 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:59.780404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1138/2263] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:15:59.784794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d78ae-15d1-4578-b617-154ade6f0085 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:59.795856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d78ae-15d1-4578-b617-154ade6f0085 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:15:59.800840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf270bec-bbfa-48a6-93cb-7dd9259bdbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:15:59.801573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf270bec-bbfa-48a6-93cb-7dd9259bdbf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:15:59.801885 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf270bec-bbfa-48a6-93cb-7dd9259bdbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:15:59.802204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf270bec-bbfa-48a6-93cb-7dd9259bdbf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:15:59.802944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf270bec-bbfa-48a6-93cb-7dd9259bdbf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:15:59.804854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf270bec-bbfa-48a6-93cb-7dd9259bdbf0 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:15:59.963244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1126/2264] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:15:59 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:15:59.964289 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e068035-ab54-4b76-934b-96d174296f19 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:15:59.975025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e068035-ab54-4b76-934b-96d174296f19 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:00.008259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:00.032670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb81bd-d28e-4ecf-b07e-d2e136aded18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:00.033457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb81bd-d28e-4ecf-b07e-d2e136aded18 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:00.033877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb81bd-d28e-4ecf-b07e-d2e136aded18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:00.034252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb81bd-d28e-4ecf-b07e-d2e136aded18 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:00.035029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb81bd-d28e-4ecf-b07e-d2e136aded18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:00.037223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb81bd-d28e-4ecf-b07e-d2e136aded18 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:00.155775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1139/2265] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:15:59 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:16:00.158490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf270bec-bbfa-48a6-93cb-7dd9259bdbf0 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:00.169684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf270bec-bbfa-48a6-93cb-7dd9259bdbf0 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:00.174478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c4a46-7835-4bec-aafb-125881521af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:00.175119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c4a46-7835-4bec-aafb-125881521af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:00.175383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c4a46-7835-4bec-aafb-125881521af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:00.175662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c4a46-7835-4bec-aafb-125881521af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:00.176393 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c4a46-7835-4bec-aafb-125881521af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:00.178010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c4a46-7835-4bec-aafb-125881521af0 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:00.190284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1127/2266] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:00.198724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb81bd-d28e-4ecf-b07e-d2e136aded18 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:00.209781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb81bd-d28e-4ecf-b07e-d2e136aded18 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:00.214612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b449d37-5e3e-4ebd-b329-fd64dd2490a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:00.215155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b449d37-5e3e-4ebd-b329-fd64dd2490a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:00.215438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b449d37-5e3e-4ebd-b329-fd64dd2490a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:00.215798 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b449d37-5e3e-4ebd-b329-fd64dd2490a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:00.216652 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b449d37-5e3e-4ebd-b329-fd64dd2490a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:00.218284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b449d37-5e3e-4ebd-b329-fd64dd2490a9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:00.232490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1140/2267] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:00.236797 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c4a46-7835-4bec-aafb-125881521af0 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:00.247571 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c4a46-7835-4bec-aafb-125881521af0 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:00.252490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd2e02-8104-4dcd-99a6-6b6322fdeed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:00.253376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd2e02-8104-4dcd-99a6-6b6322fdeed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:00.253681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd2e02-8104-4dcd-99a6-6b6322fdeed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:00.266837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd2e02-8104-4dcd-99a6-6b6322fdeed3 admin admin] PATH_INFO: `/v3/projects/4b5ccae799cd4d20a9ddef31d700f625/users/336592cddf7e46d3aef701cb11af0807/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:00.277205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd2e02-8104-4dcd-99a6-6b6322fdeed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5ccae799cd4d20a9ddef31d700f625, user_id=336592cddf7e46d3aef701cb11af0807, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:00.280334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd2e02-8104-4dcd-99a6-6b6322fdeed3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:00.363120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:00.370954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1128/2268] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:00 2022] PUT /identity/v3/projects/4b5ccae799cd4d20a9ddef31d700f625/users/336592cddf7e46d3aef701cb11af0807/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:00.387153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af156d66-54e2-4a83-bcf9-b3e761e14f75 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:00.388059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af156d66-54e2-4a83-bcf9-b3e761e14f75 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:00.388742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af156d66-54e2-4a83-bcf9-b3e761e14f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:00.425560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-af156d66-54e2-4a83-bcf9-b3e761e14f75 None None] MFA Rules not processed for user `336592cddf7e46d3aef701cb11af0807`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:00.426223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-af156d66-54e2-4a83-bcf9-b3e761e14f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:00.500547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1141/2269] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:16:00.512119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd2e02-8104-4dcd-99a6-6b6322fdeed3 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:00.514313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c1bd2e02-8104-4dcd-99a6-6b6322fdeed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:00.580563 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd2e02-8104-4dcd-99a6-6b6322fdeed3 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:00.587347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c8ff8f-8dd2-4267-aa32-37645a9c89f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:00.588339 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8ff8f-8dd2-4267-aa32-37645a9c89f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:00.589757 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8ff8f-8dd2-4267-aa32-37645a9c89f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:00.590995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8ff8f-8dd2-4267-aa32-37645a9c89f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:00.605287 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c8ff8f-8dd2-4267-aa32-37645a9c89f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:00.606754 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c8ff8f-8dd2-4267-aa32-37645a9c89f5 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:00.617556 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1129/2270] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:01.952412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af156d66-54e2-4a83-bcf9-b3e761e14f75 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:01.953898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-af156d66-54e2-4a83-bcf9-b3e761e14f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:02.021039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af156d66-54e2-4a83-bcf9-b3e761e14f75 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:02.025541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:02.026197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:02.026480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:02.026764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] PATH_INFO: `/v3/users/6f9c3013799c40eea114118a96c66740` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:02.027946 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f9c3013799c40eea114118a96c66740)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:02.029992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:02.233357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:02.234985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f9c3013799c40eea114118a96c66740'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:02.235661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f9c3013799c40eea114118a96c66740'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:02.243055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f9c3013799c40eea114118a96c66740'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:02.352440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f9c3013799c40eea114118a96c66740'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:02.355706 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f9c3013799c40eea114118a96c66740'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:02.374231 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1142/2271] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:01 2022] DELETE /identity/v3/users/6f9c3013799c40eea114118a96c66740 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:02.628915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c8ff8f-8dd2-4267-aa32-37645a9c89f5 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:02.642745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c8ff8f-8dd2-4267-aa32-37645a9c89f5 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:02.721675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:02.722325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:02.722606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:02.722881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] PATH_INFO: `/v3/projects/1e064b693a124655b12e8987a9948717` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:02.724681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e064b693a124655b12e8987a9948717)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:02.726637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:02.854886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e064b693a124655b12e8987a9948717', 'request_id': 'req-2d2c437c-1a37-4005-ae90-a356cc116668'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:02.860409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e064b693a124655b12e8987a9948717', 'request_id': 'req-2d2c437c-1a37-4005-ae90-a356cc116668'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:02.951789 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e064b693a124655b12e8987a9948717', 'request_id': 'req-2d2c437c-1a37-4005-ae90-a356cc116668'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:02.964162 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] The token cache is being invalidate because project 1e064b693a124655b12e8987a9948717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:02.965888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:02.965888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:02.967015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1130/2272] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:02 2022] DELETE /identity/v3/projects/1e064b693a124655b12e8987a9948717 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:16:02.970925 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:02.972408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:02.986207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82d8a1-8237-4fdf-ac3d-78261ab6eefb admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:03.043647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:03.044297 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:03.044582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:03.044862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] PATH_INFO: `/v3/users/ce3e50180d624207bd7d59b766236ec7` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:03.046264 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce3e50180d624207bd7d59b766236ec7)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:03.048346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:03.267021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce3e50180d624207bd7d59b766236ec7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:03.267869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce3e50180d624207bd7d59b766236ec7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:03.277765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce3e50180d624207bd7d59b766236ec7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:03.368195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce3e50180d624207bd7d59b766236ec7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:03.368839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce3e50180d624207bd7d59b766236ec7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:03.387296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1143/2273] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:02 2022] DELETE /identity/v3/users/ce3e50180d624207bd7d59b766236ec7 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:03.633669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:03.635663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:03.649352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c437c-1a37-4005-ae90-a356cc116668 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:03.685524 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:03.696756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:03.698403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:03.718157 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:03.720093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:03.720093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:03.720093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] PATH_INFO: `/v3/projects/72a3e387c6064e06847dcd92b46be4c3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:03.721356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72a3e387c6064e06847dcd92b46be4c3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:03.724674 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:03.765752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:03.789976 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0bf29-21d0-4f51-b816-b76319b965f2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:03.794853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f93d6-7689-4135-bb5d-93a598d0ef55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:03.795514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f93d6-7689-4135-bb5d-93a598d0ef55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:03.795801 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f93d6-7689-4135-bb5d-93a598d0ef55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:03.796091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f93d6-7689-4135-bb5d-93a598d0ef55 admin admin] PATH_INFO: `/v3/users/9c0acd2da8d240f4a6d41d1fc0ba8816` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:03.797427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f93d6-7689-4135-bb5d-93a598d0ef55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c0acd2da8d240f4a6d41d1fc0ba8816)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:03.799414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f93d6-7689-4135-bb5d-93a598d0ef55 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:03.861605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72a3e387c6064e06847dcd92b46be4c3', 'request_id': 'req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:03.867774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72a3e387c6064e06847dcd92b46be4c3', 'request_id': 'req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:04.003535 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72a3e387c6064e06847dcd92b46be4c3', 'request_id': 'req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:04.015461 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] The token cache is being invalidate because project 72a3e387c6064e06847dcd92b46be4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:04.015753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:04.016459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-da00e0f5-57b5-4372-b7a7-dc8a5de6f9f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:04.018016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1131/2274] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:03 2022] DELETE /identity/v3/projects/72a3e387c6064e06847dcd92b46be4c3 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:04.022791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49e7f7-67c8-4464-88f1-64a578904ae3 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.023094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49e7f7-67c8-4464-88f1-64a578904ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.023374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49e7f7-67c8-4464-88f1-64a578904ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.044170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1e49e7f7-67c8-4464-88f1-64a578904ae3 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:04.045423 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1e49e7f7-67c8-4464-88f1-64a578904ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:04.094659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f03f93d6-7689-4135-bb5d-93a598d0ef55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c0acd2da8d240f4a6d41d1fc0ba8816'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:04.095289 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f03f93d6-7689-4135-bb5d-93a598d0ef55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c0acd2da8d240f4a6d41d1fc0ba8816'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:04.103515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f03f93d6-7689-4135-bb5d-93a598d0ef55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c0acd2da8d240f4a6d41d1fc0ba8816'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:04.115435 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1132/2275] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:16:04.123483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5e716-30da-4200-a08e-cdab181ee5b8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.123838 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5e716-30da-4200-a08e-cdab181ee5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.124128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5e716-30da-4200-a08e-cdab181ee5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.144741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-aeb5e716-30da-4200-a08e-cdab181ee5b8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:04.146489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f03f93d6-7689-4135-bb5d-93a598d0ef55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c0acd2da8d240f4a6d41d1fc0ba8816'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:04.146632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-aeb5e716-30da-4200-a08e-cdab181ee5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:04.147222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f03f93d6-7689-4135-bb5d-93a598d0ef55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c0acd2da8d240f4a6d41d1fc0ba8816'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:04.154752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1133/2276] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:16:04.158864 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5e716-30da-4200-a08e-cdab181ee5b8 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:04.168906 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5e716-30da-4200-a08e-cdab181ee5b8 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:04.169170 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1144/2277] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:03 2022] DELETE /identity/v3/users/9c0acd2da8d240f4a6d41d1fc0ba8816 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:04.177381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5adfe8-e08e-4358-9d9c-97f4fd7f6440 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.177792 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5adfe8-e08e-4358-9d9c-97f4fd7f6440 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.178072 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5adfe8-e08e-4358-9d9c-97f4fd7f6440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.213843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-da5adfe8-e08e-4358-9d9c-97f4fd7f6440 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:04.215208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-da5adfe8-e08e-4358-9d9c-97f4fd7f6440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:04.234094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:04.242266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b1e3b-bd1e-4a3c-9b21-9d7289bc200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:04.242891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b1e3b-bd1e-4a3c-9b21-9d7289bc200d admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.243173 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b1e3b-bd1e-4a3c-9b21-9d7289bc200d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.243490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b1e3b-bd1e-4a3c-9b21-9d7289bc200d admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.244386 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b1e3b-bd1e-4a3c-9b21-9d7289bc200d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:16:04.244827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b1e3b-bd1e-4a3c-9b21-9d7289bc200d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:04.247089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b1e3b-bd1e-4a3c-9b21-9d7289bc200d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:04.260110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1134/2278] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:16:04.266083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b1e3b-bd1e-4a3c-9b21-9d7289bc200d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:04.276830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b1e3b-bd1e-4a3c-9b21-9d7289bc200d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:04.281943 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d816e-0165-44c6-ab91-5c6322cb4e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:04.282557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d816e-0165-44c6-ab91-5c6322cb4e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.282837 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d816e-0165-44c6-ab91-5c6322cb4e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.283111 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d816e-0165-44c6-ab91-5c6322cb4e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.283779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d816e-0165-44c6-ab91-5c6322cb4e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:04.286385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d816e-0165-44c6-ab91-5c6322cb4e45 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:04.288222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1145/2279] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:16:04.298376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5adfe8-e08e-4358-9d9c-97f4fd7f6440 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:04.300130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-da5adfe8-e08e-4358-9d9c-97f4fd7f6440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:04.365884 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5adfe8-e08e-4358-9d9c-97f4fd7f6440 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:04.370732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c923c5-d516-4c61-a900-f2594109b9f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:04.371367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c923c5-d516-4c61-a900-f2594109b9f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.371649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c923c5-d516-4c61-a900-f2594109b9f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.371925 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c923c5-d516-4c61-a900-f2594109b9f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.384093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c923c5-d516-4c61-a900-f2594109b9f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:04.385857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c923c5-d516-4c61-a900-f2594109b9f6 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:04.393548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1135/2280] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:16:04 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:16:04.394447 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d816e-0165-44c6-ab91-5c6322cb4e45 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:04.407549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1146/2281] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:16:04.467813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d816e-0165-44c6-ab91-5c6322cb4e45 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:04.473219 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd33b1-8f62-4cdf-93a2-db648a7d306d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:04.473854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd33b1-8f62-4cdf-93a2-db648a7d306d admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.474144 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd33b1-8f62-4cdf-93a2-db648a7d306d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.474425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd33b1-8f62-4cdf-93a2-db648a7d306d admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.475232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd33b1-8f62-4cdf-93a2-db648a7d306d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:04.477137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd33b1-8f62-4cdf-93a2-db648a7d306d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:04.547625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1136/2282] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:16:04 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:04.551909 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c923c5-d516-4c61-a900-f2594109b9f6 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:04.564019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:04.573602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c923c5-d516-4c61-a900-f2594109b9f6 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:04.581066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda1688-8c0c-49c6-b9ce-fec77561ebb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:04.581690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda1688-8c0c-49c6-b9ce-fec77561ebb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.582125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda1688-8c0c-49c6-b9ce-fec77561ebb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.582226 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda1688-8c0c-49c6-b9ce-fec77561ebb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.583588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda1688-8c0c-49c6-b9ce-fec77561ebb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:04.585195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda1688-8c0c-49c6-b9ce-fec77561ebb4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:04.597175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1147/2283] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:16:04.615161 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd33b1-8f62-4cdf-93a2-db648a7d306d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:04.625905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd33b1-8f62-4cdf-93a2-db648a7d306d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:04.631903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530c0cf-3b2d-4a25-95a7-0cb72c09b565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:04.632570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530c0cf-3b2d-4a25-95a7-0cb72c09b565 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.633002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530c0cf-3b2d-4a25-95a7-0cb72c09b565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.633156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530c0cf-3b2d-4a25-95a7-0cb72c09b565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.633959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530c0cf-3b2d-4a25-95a7-0cb72c09b565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:04.635714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530c0cf-3b2d-4a25-95a7-0cb72c09b565 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:04.649022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1137/2284] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:04.652648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda1688-8c0c-49c6-b9ce-fec77561ebb4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:04.663048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda1688-8c0c-49c6-b9ce-fec77561ebb4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:04.667847 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f02cb-9399-48be-a08c-6407a64485aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:04.668517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f02cb-9399-48be-a08c-6407a64485aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.668823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f02cb-9399-48be-a08c-6407a64485aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.669197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f02cb-9399-48be-a08c-6407a64485aa admin admin] PATH_INFO: `/v3/projects/1d81962138ca49fcb8d8e6f3776ec4b6/users/e7ec1dc1e477434c99abb082dbf52deb/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.693922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f02cb-9399-48be-a08c-6407a64485aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d81962138ca49fcb8d8e6f3776ec4b6, user_id=e7ec1dc1e477434c99abb082dbf52deb, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:04.697138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f02cb-9399-48be-a08c-6407a64485aa admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:04.787254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1148/2285] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:04 2022] PUT /identity/v3/projects/1d81962138ca49fcb8d8e6f3776ec4b6/users/e7ec1dc1e477434c99abb082dbf52deb/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:04.799399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56a243-044e-4d33-b821-9e07ce3d5d2e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.799691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56a243-044e-4d33-b821-9e07ce3d5d2e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.799974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56a243-044e-4d33-b821-9e07ce3d5d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.828832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3c56a243-044e-4d33-b821-9e07ce3d5d2e None None] MFA Rules not processed for user `e7ec1dc1e477434c99abb082dbf52deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:04.830095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3c56a243-044e-4d33-b821-9e07ce3d5d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:04.896813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1138/2286] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:04.901690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f02cb-9399-48be-a08c-6407a64485aa admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:04.959914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56a243-044e-4d33-b821-9e07ce3d5d2e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:04.961574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3c56a243-044e-4d33-b821-9e07ce3d5d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:04.964238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f02cb-9399-48be-a08c-6407a64485aa admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:04.969527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f38bf-b080-4187-b608-3bb011e9f49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:04.971928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f38bf-b080-4187-b608-3bb011e9f49d admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.971928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f38bf-b080-4187-b608-3bb011e9f49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.971928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f38bf-b080-4187-b608-3bb011e9f49d admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.971928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f38bf-b080-4187-b608-3bb011e9f49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:04.974367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f38bf-b080-4187-b608-3bb011e9f49d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:04.980338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56a243-044e-4d33-b821-9e07ce3d5d2e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:04.985680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:04.986548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:04.986668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:04.986893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] PATH_INFO: `/v3/projects/1b309c8941ae48b4864a368001da0679` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:04.988228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b309c8941ae48b4864a368001da0679)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:04.993789 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:05.067417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1149/2287] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:16:04 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:05.071351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f38bf-b080-4187-b608-3bb011e9f49d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:05.088154 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:05.101956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b309c8941ae48b4864a368001da0679', 'request_id': 'req-c3215488-6a9a-4afb-b593-536ead1c33c6'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:05.108076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b309c8941ae48b4864a368001da0679', 'request_id': 'req-c3215488-6a9a-4afb-b593-536ead1c33c6'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:05.137775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f38bf-b080-4187-b608-3bb011e9f49d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:05.162488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b309c8941ae48b4864a368001da0679', 'request_id': 'req-c3215488-6a9a-4afb-b593-536ead1c33c6'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:05.174624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] The token cache is being invalidate because project 1b309c8941ae48b4864a368001da0679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:05.174917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:05.175583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:05.177320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1139/2288] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:04 2022] DELETE /identity/v3/projects/1b309c8941ae48b4864a368001da0679 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:16:05.181419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:05.182832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:05.194943 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fbf12-5d80-4278-ac1c-0b762d9c63a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:05.195689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fbf12-5d80-4278-ac1c-0b762d9c63a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:05.196014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fbf12-5d80-4278-ac1c-0b762d9c63a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:05.196313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fbf12-5d80-4278-ac1c-0b762d9c63a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:05.197016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fbf12-5d80-4278-ac1c-0b762d9c63a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:05.198973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fbf12-5d80-4278-ac1c-0b762d9c63a3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:05.247767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3215488-6a9a-4afb-b593-536ead1c33c6 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:05.252561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:05.253262 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:05.253594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:05.253875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] PATH_INFO: `/v3/users/1dd93f55bafe499bb07e309a95fefb4f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:05.255551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dd93f55bafe499bb07e309a95fefb4f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:05.257632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:05.275494 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1150/2289] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:16:05 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:16:05.276618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fbf12-5d80-4278-ac1c-0b762d9c63a3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:05.278334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5c1fbf12-5d80-4278-ac1c-0b762d9c63a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:05.291134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fbf12-5d80-4278-ac1c-0b762d9c63a3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:05.296246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8ff31-5013-4df8-a5ba-a2ea1a355d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:05.296865 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8ff31-5013-4df8-a5ba-a2ea1a355d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:05.297184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8ff31-5013-4df8-a5ba-a2ea1a355d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:05.297468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8ff31-5013-4df8-a5ba-a2ea1a355d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:05.298173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8ff31-5013-4df8-a5ba-a2ea1a355d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:05.299776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8ff31-5013-4df8-a5ba-a2ea1a355d97 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:05.313128 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1151/2290] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:05.314102 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8ff31-5013-4df8-a5ba-a2ea1a355d97 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:05.324455 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8ff31-5013-4df8-a5ba-a2ea1a355d97 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:05.330099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3311b-cff8-4ec5-8d99-1f8fbb18ae38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:05.330790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3311b-cff8-4ec5-8d99-1f8fbb18ae38 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:05.335350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3311b-cff8-4ec5-8d99-1f8fbb18ae38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:05.335350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3311b-cff8-4ec5-8d99-1f8fbb18ae38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:05.335350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3311b-cff8-4ec5-8d99-1f8fbb18ae38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:05.335350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3311b-cff8-4ec5-8d99-1f8fbb18ae38 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:05.344914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1152/2291] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:16:05.348760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3311b-cff8-4ec5-8d99-1f8fbb18ae38 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:05.358202 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3311b-cff8-4ec5-8d99-1f8fbb18ae38 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:05.362952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dedc6b6-3830-4bd8-a30f-7929a5f902ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:05.363612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedc6b6-3830-4bd8-a30f-7929a5f902ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:05.367612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedc6b6-3830-4bd8-a30f-7929a5f902ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:05.367612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedc6b6-3830-4bd8-a30f-7929a5f902ae admin admin] PATH_INFO: `/v3/projects/d9ab80831f2642719c413e13041dd6ea/users/af43ff0dd69f40f7a151dc25992836a7/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:05.378207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dedc6b6-3830-4bd8-a30f-7929a5f902ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ab80831f2642719c413e13041dd6ea, user_id=af43ff0dd69f40f7a151dc25992836a7, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:05.381481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dedc6b6-3830-4bd8-a30f-7929a5f902ae admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:05.458017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dd93f55bafe499bb07e309a95fefb4f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:05.464830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dd93f55bafe499bb07e309a95fefb4f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:05.478113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1153/2292] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:05 2022] PUT /identity/v3/projects/d9ab80831f2642719c413e13041dd6ea/users/af43ff0dd69f40f7a151dc25992836a7/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:16:05.488328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcf37c-7292-4c91-bab7-efe7b36c6844 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:05.488636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcf37c-7292-4c91-bab7-efe7b36c6844 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:05.488913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcf37c-7292-4c91-bab7-efe7b36c6844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:05.516986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c6dcf37c-7292-4c91-bab7-efe7b36c6844 None None] MFA Rules not processed for user `af43ff0dd69f40f7a151dc25992836a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:05.518293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c6dcf37c-7292-4c91-bab7-efe7b36c6844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:05.581917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1154/2293] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:16:05.587075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcf37c-7292-4c91-bab7-efe7b36c6844 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:05.597374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dd93f55bafe499bb07e309a95fefb4f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:05.598218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dd93f55bafe499bb07e309a95fefb4f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:05.612354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dd93f55bafe499bb07e309a95fefb4f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:05.619483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1140/2294] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:05 2022] DELETE /identity/v3/users/1dd93f55bafe499bb07e309a95fefb4f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:16:05.657443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcf37c-7292-4c91-bab7-efe7b36c6844 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:05.733869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f413519-a26f-4aa2-94bf-06b0a5dd2313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:05.734535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f413519-a26f-4aa2-94bf-06b0a5dd2313 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:05.734820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f413519-a26f-4aa2-94bf-06b0a5dd2313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:05.735097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f413519-a26f-4aa2-94bf-06b0a5dd2313 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:05.735777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f413519-a26f-4aa2-94bf-06b0a5dd2313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:05.737712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f413519-a26f-4aa2-94bf-06b0a5dd2313 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:05.883155 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1155/2295] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:16:05 2022] POST /identity/v3/projects => generated 377 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:16:05.884481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:05.886036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:05.900173 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14184f8d-2121-48df-8af8-2b1250b03a32 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:05.970971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929183ca-2a2f-4099-ae94-278f98042f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:05.971846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929183ca-2a2f-4099-ae94-278f98042f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:05.972230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929183ca-2a2f-4099-ae94-278f98042f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:05.972634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929183ca-2a2f-4099-ae94-278f98042f30 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:05.974708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929183ca-2a2f-4099-ae94-278f98042f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:05.977541 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929183ca-2a2f-4099-ae94-278f98042f30 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.057916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1141/2296] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:16:05 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:06.062063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f413519-a26f-4aa2-94bf-06b0a5dd2313 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.073928 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f413519-a26f-4aa2-94bf-06b0a5dd2313 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.079374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0570a-acc6-4b76-a9fd-0d3ba41a4e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.079658 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0570a-acc6-4b76-a9fd-0d3ba41a4e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.079944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0570a-acc6-4b76-a9fd-0d3ba41a4e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.080223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0570a-acc6-4b76-a9fd-0d3ba41a4e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.080931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0570a-acc6-4b76-a9fd-0d3ba41a4e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:06.082709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0570a-acc6-4b76-a9fd-0d3ba41a4e86 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.093880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1156/2297] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:16:06.098686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929183ca-2a2f-4099-ae94-278f98042f30 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.109055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929183ca-2a2f-4099-ae94-278f98042f30 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.112982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:06.114719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd26de-5e08-4a1f-8518-52efe7c00c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.115402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd26de-5e08-4a1f-8518-52efe7c00c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.115700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd26de-5e08-4a1f-8518-52efe7c00c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.115979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd26de-5e08-4a1f-8518-52efe7c00c07 admin admin] PATH_INFO: `/v3/projects/dfdab63c1ace48ff9608722b5b695bc7/users/d6fd953955c54fad8ebe376b64c16dcd/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.127514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0570a-acc6-4b76-a9fd-0d3ba41a4e86 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.131198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-76d0570a-acc6-4b76-a9fd-0d3ba41a4e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:06.143195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd26de-5e08-4a1f-8518-52efe7c00c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfdab63c1ace48ff9608722b5b695bc7, user_id=d6fd953955c54fad8ebe376b64c16dcd, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:06.146311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd26de-5e08-4a1f-8518-52efe7c00c07 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.147868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0570a-acc6-4b76-a9fd-0d3ba41a4e86 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.154321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.154973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.155410 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.155695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] PATH_INFO: `/v3/projects/3a1f5e633db8492e9f6f3ec06926b6cc` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.157388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a1f5e633db8492e9f6f3ec06926b6cc)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:06.159670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.220071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1142/2298] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:06 2022] PUT /identity/v3/projects/dfdab63c1ace48ff9608722b5b695bc7/users/d6fd953955c54fad8ebe376b64c16dcd/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:06.224424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd26de-5e08-4a1f-8518-52efe7c00c07 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.290613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd26de-5e08-4a1f-8518-52efe7c00c07 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.302805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a1f5e633db8492e9f6f3ec06926b6cc', 'request_id': 'req-87987905-a896-4251-83b0-f42769dc6864'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:06.308596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a1f5e633db8492e9f6f3ec06926b6cc', 'request_id': 'req-87987905-a896-4251-83b0-f42769dc6864'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:06.315254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a1f5e633db8492e9f6f3ec06926b6cc', 'request_id': 'req-87987905-a896-4251-83b0-f42769dc6864'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:06.353661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7629919-0fac-4311-96cf-e7eec9bacf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.354285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7629919-0fac-4311-96cf-e7eec9bacf41 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.354567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7629919-0fac-4311-96cf-e7eec9bacf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.354841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7629919-0fac-4311-96cf-e7eec9bacf41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.355612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7629919-0fac-4311-96cf-e7eec9bacf41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:06.357370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7629919-0fac-4311-96cf-e7eec9bacf41 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.371159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1143/2299] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:16:06.372178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7629919-0fac-4311-96cf-e7eec9bacf41 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.382129 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7629919-0fac-4311-96cf-e7eec9bacf41 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.386860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17f7e1-af9c-4904-b2b3-03ecee6c6b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.387514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17f7e1-af9c-4904-b2b3-03ecee6c6b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.387798 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17f7e1-af9c-4904-b2b3-03ecee6c6b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.388094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17f7e1-af9c-4904-b2b3-03ecee6c6b4a admin admin] PATH_INFO: `/v3/projects/dfdab63c1ace48ff9608722b5b695bc7/users/d6fd953955c54fad8ebe376b64c16dcd/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.390082 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17f7e1-af9c-4904-b2b3-03ecee6c6b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfdab63c1ace48ff9608722b5b695bc7, user_id=d6fd953955c54fad8ebe376b64c16dcd, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:06.392770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17f7e1-af9c-4904-b2b3-03ecee6c6b4a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.403665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1144/2300] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:06 2022] PUT /identity/v3/projects/dfdab63c1ace48ff9608722b5b695bc7/users/d6fd953955c54fad8ebe376b64c16dcd/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:16:06.408947 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] The token cache is being invalidate because project 3a1f5e633db8492e9f6f3ec06926b6cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:06.409309 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:06.409963 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-87987905-a896-4251-83b0-f42769dc6864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:06.412497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1157/2301] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:06 2022] DELETE /identity/v3/projects/3a1f5e633db8492e9f6f3ec06926b6cc => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:06.417811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ae2df-5d73-4977-8e24-18c9b6ff554e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.419163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a0c6e-33c7-41c5-831b-76c018e543a8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.419163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a0c6e-33c7-41c5-831b-76c018e543a8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.419163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a0c6e-33c7-41c5-831b-76c018e543a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.419705 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ae2df-5d73-4977-8e24-18c9b6ff554e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.423709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ae2df-5d73-4977-8e24-18c9b6ff554e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.442574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-ee4ae2df-5d73-4977-8e24-18c9b6ff554e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:06.443966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ee4ae2df-5d73-4977-8e24-18c9b6ff554e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:06.458999 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c10a0c6e-33c7-41c5-831b-76c018e543a8 None None] MFA Rules not processed for user `d6fd953955c54fad8ebe376b64c16dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:06.460306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c10a0c6e-33c7-41c5-831b-76c018e543a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:06.523576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1145/2302] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:06.525761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c166f1a-c78f-49d3-b0c2-580bb3f1091a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.526266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c166f1a-c78f-49d3-b0c2-580bb3f1091a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.526611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c166f1a-c78f-49d3-b0c2-580bb3f1091a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.554382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1158/2303] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:06.555270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1c166f1a-c78f-49d3-b0c2-580bb3f1091a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:06.557248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1c166f1a-c78f-49d3-b0c2-580bb3f1091a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:06.565901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a0c6e-33c7-41c5-831b-76c018e543a8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.567575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1146/2304] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:06.568065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c10a0c6e-33c7-41c5-831b-76c018e543a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:06.574006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c166f1a-c78f-49d3-b0c2-580bb3f1091a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.583998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c166f1a-c78f-49d3-b0c2-580bb3f1091a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.588982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7c20c-c3c9-430b-a3a2-404d6722e4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.589611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c20c-c3c9-430b-a3a2-404d6722e4ac admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.589894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c20c-c3c9-430b-a3a2-404d6722e4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.590632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c20c-c3c9-430b-a3a2-404d6722e4ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.591775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7c20c-c3c9-430b-a3a2-404d6722e4ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:16:06.592090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7c20c-c3c9-430b-a3a2-404d6722e4ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:06.593762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7c20c-c3c9-430b-a3a2-404d6722e4ac admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.606844 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1147/2305] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:16:06.611278 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7c20c-c3c9-430b-a3a2-404d6722e4ac admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.621254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7c20c-c3c9-430b-a3a2-404d6722e4ac admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.626148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7d2ac-9e35-482b-8bf4-32fd63874661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.626757 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7d2ac-9e35-482b-8bf4-32fd63874661 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.627041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7d2ac-9e35-482b-8bf4-32fd63874661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.627322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7d2ac-9e35-482b-8bf4-32fd63874661 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.627988 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7d2ac-9e35-482b-8bf4-32fd63874661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:06.629954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7d2ac-9e35-482b-8bf4-32fd63874661 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.634517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a0c6e-33c7-41c5-831b-76c018e543a8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.638249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401861-6b75-4996-9912-b92accc0a2e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.638873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401861-6b75-4996-9912-b92accc0a2e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.642286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401861-6b75-4996-9912-b92accc0a2e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.642286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401861-6b75-4996-9912-b92accc0a2e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.642286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4c401861-6b75-4996-9912-b92accc0a2e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:06.704549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c401861-6b75-4996-9912-b92accc0a2e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:06.706462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c401861-6b75-4996-9912-b92accc0a2e3 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.708603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1148/2306] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:16:06 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:06.712641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7d2ac-9e35-482b-8bf4-32fd63874661 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.718530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1159/2307] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:16:06.747382 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:06.781010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7d2ac-9e35-482b-8bf4-32fd63874661 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.787374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074528d0-c72a-46a4-bac3-a4f165d92ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.788376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074528d0-c72a-46a4-bac3-a4f165d92ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.789092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074528d0-c72a-46a4-bac3-a4f165d92ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.789092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074528d0-c72a-46a4-bac3-a4f165d92ac3 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.790005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074528d0-c72a-46a4-bac3-a4f165d92ac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:06.792259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074528d0-c72a-46a4-bac3-a4f165d92ac3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.889472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1149/2308] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:16:06 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:16:06.893709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401861-6b75-4996-9912-b92accc0a2e3 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.904341 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401861-6b75-4996-9912-b92accc0a2e3 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.908956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2c08c-d49a-4fe0-9535-bf18fa535678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.909605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2c08c-d49a-4fe0-9535-bf18fa535678 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.909890 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2c08c-d49a-4fe0-9535-bf18fa535678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.910342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2c08c-d49a-4fe0-9535-bf18fa535678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.911384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2c08c-d49a-4fe0-9535-bf18fa535678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:06.913001 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2c08c-d49a-4fe0-9535-bf18fa535678 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.923000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1160/2309] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:06.927594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074528d0-c72a-46a4-bac3-a4f165d92ac3 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.937820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074528d0-c72a-46a4-bac3-a4f165d92ac3 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.943259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39c03b-77be-4fe2-bb60-a64193adc1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.943886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39c03b-77be-4fe2-bb60-a64193adc1bf admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.944171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39c03b-77be-4fe2-bb60-a64193adc1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.944452 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39c03b-77be-4fe2-bb60-a64193adc1bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:06.945194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39c03b-77be-4fe2-bb60-a64193adc1bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:06.946924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39c03b-77be-4fe2-bb60-a64193adc1bf admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:06.957569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1150/2310] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:06.961724 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2c08c-d49a-4fe0-9535-bf18fa535678 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:06.972066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2c08c-d49a-4fe0-9535-bf18fa535678 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:06.977542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e29450-4b51-4554-a393-bd5ecbe14615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:06.978293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e29450-4b51-4554-a393-bd5ecbe14615 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:06.978574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e29450-4b51-4554-a393-bd5ecbe14615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:06.978858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e29450-4b51-4554-a393-bd5ecbe14615 admin admin] PATH_INFO: `/v3/projects/9a5ef7d3df15467da97b3b7d352c6f34/users/a5340cff2c564fe1b7d7cc6f6ac8f95d/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:07.005386 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e29450-4b51-4554-a393-bd5ecbe14615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a5ef7d3df15467da97b3b7d352c6f34, user_id=a5340cff2c564fe1b7d7cc6f6ac8f95d, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:07.008448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e29450-4b51-4554-a393-bd5ecbe14615 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:07.116079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1161/2311] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:06 2022] PUT /identity/v3/projects/9a5ef7d3df15467da97b3b7d352c6f34/users/a5340cff2c564fe1b7d7cc6f6ac8f95d/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:16:07.117550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730066e-f542-4850-ba2d-e22105e53e7e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:07.117866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730066e-f542-4850-ba2d-e22105e53e7e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:07.118134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730066e-f542-4850-ba2d-e22105e53e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:07.154526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6730066e-f542-4850-ba2d-e22105e53e7e None None] MFA Rules not processed for user `a5340cff2c564fe1b7d7cc6f6ac8f95d`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:07.156089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6730066e-f542-4850-ba2d-e22105e53e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:07.231664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1151/2312] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:16:07.236235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e29450-4b51-4554-a393-bd5ecbe14615 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:07.273857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:07.302182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e29450-4b51-4554-a393-bd5ecbe14615 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:07.307109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf3db3-e85b-4bd4-82be-6dc48bb54d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:07.307728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf3db3-e85b-4bd4-82be-6dc48bb54d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:07.307998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf3db3-e85b-4bd4-82be-6dc48bb54d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:07.308272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf3db3-e85b-4bd4-82be-6dc48bb54d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:07.308958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf3db3-e85b-4bd4-82be-6dc48bb54d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:07.311042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf3db3-e85b-4bd4-82be-6dc48bb54d60 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:07.387277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1162/2313] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:16:07 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:07.391133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730066e-f542-4850-ba2d-e22105e53e7e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:07.402711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730066e-f542-4850-ba2d-e22105e53e7e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:07.463013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6c58d-142b-47db-aa17-99ceb77bae1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:07.463638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6c58d-142b-47db-aa17-99ceb77bae1e admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:07.463918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6c58d-142b-47db-aa17-99ceb77bae1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:07.464218 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6c58d-142b-47db-aa17-99ceb77bae1e admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:07.464889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6c58d-142b-47db-aa17-99ceb77bae1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:07.467121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6c58d-142b-47db-aa17-99ceb77bae1e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:07.586911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1152/2314] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:16:07 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:16:07.591058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf3db3-e85b-4bd4-82be-6dc48bb54d60 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:07.602529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf3db3-e85b-4bd4-82be-6dc48bb54d60 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:07.609112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296405f-4c08-48f8-a038-babbeeec9728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:07.609740 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296405f-4c08-48f8-a038-babbeeec9728 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:07.611836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296405f-4c08-48f8-a038-babbeeec9728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:07.611836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296405f-4c08-48f8-a038-babbeeec9728 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:07.611836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296405f-4c08-48f8-a038-babbeeec9728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:07.612722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296405f-4c08-48f8-a038-babbeeec9728 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:07.624138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1163/2315] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:07.628366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6c58d-142b-47db-aa17-99ceb77bae1e admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:07.638892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6c58d-142b-47db-aa17-99ceb77bae1e admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:07.643751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da387e-6c29-447c-b37e-f1bc84767221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:07.644423 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da387e-6c29-447c-b37e-f1bc84767221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:07.644717 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da387e-6c29-447c-b37e-f1bc84767221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:07.645031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da387e-6c29-447c-b37e-f1bc84767221 admin admin] PATH_INFO: `/v3/projects/bbd6ebddcda84804936386ffeff017c6/users/f2739385372b484f9d08908c571fea0a/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:07.660116 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da387e-6c29-447c-b37e-f1bc84767221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd6ebddcda84804936386ffeff017c6, user_id=f2739385372b484f9d08908c571fea0a, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:07.663321 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da387e-6c29-447c-b37e-f1bc84767221 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:07.774300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1153/2316] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:07 2022] PUT /identity/v3/projects/bbd6ebddcda84804936386ffeff017c6/users/f2739385372b484f9d08908c571fea0a/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:07.778060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296405f-4c08-48f8-a038-babbeeec9728 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:07.788641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296405f-4c08-48f8-a038-babbeeec9728 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:07.844514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64451b5-74ca-4242-b2c8-2b8bd324e78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:07.845179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64451b5-74ca-4242-b2c8-2b8bd324e78e admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:07.845477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64451b5-74ca-4242-b2c8-2b8bd324e78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:07.845757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64451b5-74ca-4242-b2c8-2b8bd324e78e admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:07.846457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64451b5-74ca-4242-b2c8-2b8bd324e78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:07.848066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64451b5-74ca-4242-b2c8-2b8bd324e78e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:07.858462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1164/2317] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:07.862809 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da387e-6c29-447c-b37e-f1bc84767221 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:07.873199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da387e-6c29-447c-b37e-f1bc84767221 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:07.878944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ff462-7883-4a1e-8add-c167a5046e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:07.878944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ff462-7883-4a1e-8add-c167a5046e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:07.878944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ff462-7883-4a1e-8add-c167a5046e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:07.879241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ff462-7883-4a1e-8add-c167a5046e72 admin admin] PATH_INFO: `/v3/projects/bbd6ebddcda84804936386ffeff017c6/users/f2739385372b484f9d08908c571fea0a/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:07.881170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ff462-7883-4a1e-8add-c167a5046e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd6ebddcda84804936386ffeff017c6, user_id=f2739385372b484f9d08908c571fea0a, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:07.883909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ff462-7883-4a1e-8add-c167a5046e72 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:07.894528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1154/2318] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:07 2022] PUT /identity/v3/projects/bbd6ebddcda84804936386ffeff017c6/users/f2739385372b484f9d08908c571fea0a/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:16:07.906315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790305d3-40b8-47cc-b758-0277ad9cec78 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:07.906628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790305d3-40b8-47cc-b758-0277ad9cec78 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:07.906882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790305d3-40b8-47cc-b758-0277ad9cec78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:07.936586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-790305d3-40b8-47cc-b758-0277ad9cec78 None None] MFA Rules not processed for user `f2739385372b484f9d08908c571fea0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:07.937894 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-790305d3-40b8-47cc-b758-0277ad9cec78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:08.009728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1165/2319] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:08.023254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ff462-7883-4a1e-8add-c167a5046e72 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:08.025273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-802ff462-7883-4a1e-8add-c167a5046e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:08.087012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ff462-7883-4a1e-8add-c167a5046e72 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:08.090886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465275b-9fef-410e-8b41-69d12d44616a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:08.091770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465275b-9fef-410e-8b41-69d12d44616a service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:08.092053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465275b-9fef-410e-8b41-69d12d44616a service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:08.092327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465275b-9fef-410e-8b41-69d12d44616a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:08.104699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465275b-9fef-410e-8b41-69d12d44616a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:08.106179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465275b-9fef-410e-8b41-69d12d44616a service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:08.117174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1155/2320] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:08 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:08.301741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:08.417466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790305d3-40b8-47cc-b758-0277ad9cec78 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:08.427777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790305d3-40b8-47cc-b758-0277ad9cec78 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:08.431553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faaa0b8-d486-442b-8fd9-d866f0148bcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:08.432177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faaa0b8-d486-442b-8fd9-d866f0148bcf service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:08.432462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faaa0b8-d486-442b-8fd9-d866f0148bcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:08.432742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faaa0b8-d486-442b-8fd9-d866f0148bcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:08.483586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:08.492765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faaa0b8-d486-442b-8fd9-d866f0148bcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:08.494255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faaa0b8-d486-442b-8fd9-d866f0148bcf service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:08.503091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1166/2321] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:09.729544 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:09.835641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465275b-9fef-410e-8b41-69d12d44616a service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:09.844939 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465275b-9fef-410e-8b41-69d12d44616a service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:09.849047 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730bc0b-ad4e-46f4-8097-8495053e5b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:09.850401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730bc0b-ad4e-46f4-8097-8495053e5b56 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:09.850815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730bc0b-ad4e-46f4-8097-8495053e5b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:09.851230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730bc0b-ad4e-46f4-8097-8495053e5b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:09.853495 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d730bc0b-ad4e-46f4-8097-8495053e5b56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:09.925632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730bc0b-ad4e-46f4-8097-8495053e5b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:09.927182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730bc0b-ad4e-46f4-8097-8495053e5b56 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:09.944165 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1156/2322] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:09.946682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:10.489986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:11.684603 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faaa0b8-d486-442b-8fd9-d866f0148bcf service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:11.699736 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faaa0b8-d486-442b-8fd9-d866f0148bcf service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:11.713296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23638f21-3251-4bc1-80c2-dd9e3d7039c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:11.714292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23638f21-3251-4bc1-80c2-dd9e3d7039c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:11.714901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23638f21-3251-4bc1-80c2-dd9e3d7039c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:11.715207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23638f21-3251-4bc1-80c2-dd9e3d7039c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:11.755275 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:11.792813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23638f21-3251-4bc1-80c2-dd9e3d7039c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:11.794483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23638f21-3251-4bc1-80c2-dd9e3d7039c4 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:11.807669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1167/2323] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:18.948687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730bc0b-ad4e-46f4-8097-8495053e5b56 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:19.011761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730bc0b-ad4e-46f4-8097-8495053e5b56 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:19.016427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:19.017143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:19.017426 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:19.017701 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] PATH_INFO: `/v3/users/a5340cff2c564fe1b7d7cc6f6ac8f95d` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:19.018887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5340cff2c564fe1b7d7cc6f6ac8f95d)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:19.020886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:19.141084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5340cff2c564fe1b7d7cc6f6ac8f95d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:19.148706 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5340cff2c564fe1b7d7cc6f6ac8f95d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:19.197709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5340cff2c564fe1b7d7cc6f6ac8f95d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:19.198364 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5340cff2c564fe1b7d7cc6f6ac8f95d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:19.213224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5340cff2c564fe1b7d7cc6f6ac8f95d'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:19.219980 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1157/2324] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:18 2022] DELETE /identity/v3/users/a5340cff2c564fe1b7d7cc6f6ac8f95d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:19.225192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada1e6f-fcb2-436d-8887-39d7b29404c2 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:19.225671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada1e6f-fcb2-436d-8887-39d7b29404c2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:19.225780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada1e6f-fcb2-436d-8887-39d7b29404c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:19.244890 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-eada1e6f-fcb2-436d-8887-39d7b29404c2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:19.246189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-eada1e6f-fcb2-436d-8887-39d7b29404c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:19.304717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1168/2325] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:19.314539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:19.315996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:19.374851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1a4dd-aaad-49b0-8b09-7794c595d602 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:19.378529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f604c-ed37-434e-9eb8-7e9a34c01077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:19.379202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f604c-ed37-434e-9eb8-7e9a34c01077 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:19.379493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f604c-ed37-434e-9eb8-7e9a34c01077 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:19.379871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f604c-ed37-434e-9eb8-7e9a34c01077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:19.389731 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f604c-ed37-434e-9eb8-7e9a34c01077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:19.391206 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f604c-ed37-434e-9eb8-7e9a34c01077 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:19.402023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1158/2326] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:19.837952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada1e6f-fcb2-436d-8887-39d7b29404c2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:19.839587 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-eada1e6f-fcb2-436d-8887-39d7b29404c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:19.851957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada1e6f-fcb2-436d-8887-39d7b29404c2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:19.858623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:19.858623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:19.858623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:19.858623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] PATH_INFO: `/v3/projects/9a5ef7d3df15467da97b3b7d352c6f34` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:19.859791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a5ef7d3df15467da97b3b7d352c6f34)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:19.861771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:19.997988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a5ef7d3df15467da97b3b7d352c6f34', 'request_id': 'req-de160a61-f5c9-49e5-b118-7744aa49f07a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:20.003466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a5ef7d3df15467da97b3b7d352c6f34', 'request_id': 'req-de160a61-f5c9-49e5-b118-7744aa49f07a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:20.010552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a5ef7d3df15467da97b3b7d352c6f34', 'request_id': 'req-de160a61-f5c9-49e5-b118-7744aa49f07a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:20.186098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] The token cache is being invalidate because project 9a5ef7d3df15467da97b3b7d352c6f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:20.186351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:20.187062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:20.188607 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1169/2327] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:19 2022] DELETE /identity/v3/projects/9a5ef7d3df15467da97b3b7d352c6f34 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:20.193357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f604c-ed37-434e-9eb8-7e9a34c01077 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:20.203484 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f604c-ed37-434e-9eb8-7e9a34c01077 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:20.215893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:20.264959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:20.265580 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:20.265865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:20.266142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] PATH_INFO: `/v3/users/f2739385372b484f9d08908c571fea0a` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:20.267328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2739385372b484f9d08908c571fea0a)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:20.269288 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:20.487443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2739385372b484f9d08908c571fea0a'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:20.495075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2739385372b484f9d08908c571fea0a'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:20.585911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2739385372b484f9d08908c571fea0a'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:20.587844 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2739385372b484f9d08908c571fea0a'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:20.603235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2739385372b484f9d08908c571fea0a'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:20.617017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1159/2328] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:20 2022] DELETE /identity/v3/users/f2739385372b484f9d08908c571fea0a => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:20.818470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:20.821620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:20.843609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de160a61-f5c9-49e5-b118-7744aa49f07a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:20.928457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:20.935781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049b74c-6b91-49f2-a0c8-3891ff70bad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:20.936505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b74c-6b91-49f2-a0c8-3891ff70bad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:20.936861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b74c-6b91-49f2-a0c8-3891ff70bad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:20.937261 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b74c-6b91-49f2-a0c8-3891ff70bad5 admin admin] PATH_INFO: `/v3/users/336592cddf7e46d3aef701cb11af0807` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:20.938572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049b74c-6b91-49f2-a0c8-3891ff70bad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=336592cddf7e46d3aef701cb11af0807)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:20.940636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049b74c-6b91-49f2-a0c8-3891ff70bad5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:21.068180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:21.069795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:21.083899 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf1673-e0dd-45c7-ac22-dd567db5aecc admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:21.089530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:21.090178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.090470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.090734 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] PATH_INFO: `/v3/projects/bbd6ebddcda84804936386ffeff017c6` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.092492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd6ebddcda84804936386ffeff017c6)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:21.100452 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:21.108596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f049b74c-6b91-49f2-a0c8-3891ff70bad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336592cddf7e46d3aef701cb11af0807'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:21.109450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f049b74c-6b91-49f2-a0c8-3891ff70bad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336592cddf7e46d3aef701cb11af0807'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:21.117351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f049b74c-6b91-49f2-a0c8-3891ff70bad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336592cddf7e46d3aef701cb11af0807'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:21.197143 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f049b74c-6b91-49f2-a0c8-3891ff70bad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336592cddf7e46d3aef701cb11af0807'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:21.197791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f049b74c-6b91-49f2-a0c8-3891ff70bad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336592cddf7e46d3aef701cb11af0807'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:21.216442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd6ebddcda84804936386ffeff017c6', 'request_id': 'req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:21.217001 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1170/2329] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:20 2022] DELETE /identity/v3/users/336592cddf7e46d3aef701cb11af0807 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:21.221161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332878-15d4-4695-b27a-5b032fd10c3e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.221453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332878-15d4-4695-b27a-5b032fd10c3e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.221738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332878-15d4-4695-b27a-5b032fd10c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.223618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd6ebddcda84804936386ffeff017c6', 'request_id': 'req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:21.240473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5b332878-15d4-4695-b27a-5b032fd10c3e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:21.241790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5b332878-15d4-4695-b27a-5b032fd10c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:21.294803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd6ebddcda84804936386ffeff017c6', 'request_id': 'req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:21.303135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1171/2330] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:16:21.308292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] The token cache is being invalidate because project bbd6ebddcda84804936386ffeff017c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:21.308593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:21.309305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:21.310875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1160/2331] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:21 2022] DELETE /identity/v3/projects/bbd6ebddcda84804936386ffeff017c6 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:21.315327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:21.315661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbeb3c0-f022-4d19-94a8-2f3fde0d9000 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.315948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbeb3c0-f022-4d19-94a8-2f3fde0d9000 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.316225 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbeb3c0-f022-4d19-94a8-2f3fde0d9000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.316747 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:21.337675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ddbeb3c0-f022-4d19-94a8-2f3fde0d9000 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:21.338901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ddbeb3c0-f022-4d19-94a8-2f3fde0d9000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:21.340800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:21.347648 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1172/2332] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:16:21.355045 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880802-3203-47e4-b4bf-c40fed6bfe8a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.355342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880802-3203-47e4-b4bf-c40fed6bfe8a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.355620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880802-3203-47e4-b4bf-c40fed6bfe8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.375101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9f880802-3203-47e4-b4bf-c40fed6bfe8a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:21.376307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9f880802-3203-47e4-b4bf-c40fed6bfe8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:21.379299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ecbbc-1cb9-4e48-9e55-578f175e87eb admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:21.385330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1173/2333] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:21.389348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f880802-3203-47e4-b4bf-c40fed6bfe8a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:21.398472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f880802-3203-47e4-b4bf-c40fed6bfe8a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:21.403471 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe806-0ec1-40ab-8074-0c1b196e002b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:21.404117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe806-0ec1-40ab-8074-0c1b196e002b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.405090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe806-0ec1-40ab-8074-0c1b196e002b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.405386 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe806-0ec1-40ab-8074-0c1b196e002b admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.406104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe806-0ec1-40ab-8074-0c1b196e002b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:16:21.406419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe806-0ec1-40ab-8074-0c1b196e002b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:21.408634 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe806-0ec1-40ab-8074-0c1b196e002b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:21.424502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1174/2334] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:16:21.429558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe806-0ec1-40ab-8074-0c1b196e002b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:21.432573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81df05-0299-47f3-b2c3-bc604efdbec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:21.433255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81df05-0299-47f3-b2c3-bc604efdbec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.433544 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81df05-0299-47f3-b2c3-bc604efdbec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.433890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81df05-0299-47f3-b2c3-bc604efdbec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.435841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1d81df05-0299-47f3-b2c3-bc604efdbec8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:21.440023 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe806-0ec1-40ab-8074-0c1b196e002b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:21.445818 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53c2bb-d140-4d2f-9306-3c07e55e05db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:21.446443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53c2bb-d140-4d2f-9306-3c07e55e05db admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.446728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53c2bb-d140-4d2f-9306-3c07e55e05db admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.447007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53c2bb-d140-4d2f-9306-3c07e55e05db admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.447682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53c2bb-d140-4d2f-9306-3c07e55e05db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:21.450058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53c2bb-d140-4d2f-9306-3c07e55e05db admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:21.452472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81df05-0299-47f3-b2c3-bc604efdbec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:21.454645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81df05-0299-47f3-b2c3-bc604efdbec8 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:21.464841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1161/2335] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:16:21.559132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1175/2336] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:16:21 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:16:21.563444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81df05-0299-47f3-b2c3-bc604efdbec8 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:21.573831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81df05-0299-47f3-b2c3-bc604efdbec8 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:21.631221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d19361-ffe3-47a3-ab4c-523d51b4cb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:21.631759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d19361-ffe3-47a3-ab4c-523d51b4cb06 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.632040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d19361-ffe3-47a3-ab4c-523d51b4cb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.632313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d19361-ffe3-47a3-ab4c-523d51b4cb06 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.633001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d19361-ffe3-47a3-ab4c-523d51b4cb06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:21.635042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d19361-ffe3-47a3-ab4c-523d51b4cb06 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:21.743548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1162/2337] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:16:21 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:21.745161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53c2bb-d140-4d2f-9306-3c07e55e05db admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:21.755962 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53c2bb-d140-4d2f-9306-3c07e55e05db admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:21.760803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096af57-c11f-43db-8799-1ddeb33b348c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:21.761453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096af57-c11f-43db-8799-1ddeb33b348c admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.761815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096af57-c11f-43db-8799-1ddeb33b348c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.762101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096af57-c11f-43db-8799-1ddeb33b348c admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.762863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096af57-c11f-43db-8799-1ddeb33b348c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:21.764510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096af57-c11f-43db-8799-1ddeb33b348c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:21.774793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1176/2338] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:21.778735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d19361-ffe3-47a3-ab4c-523d51b4cb06 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:21.789634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d19361-ffe3-47a3-ab4c-523d51b4cb06 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:21.794809 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8a8f3-47ea-4236-92b2-b5ec8ecab314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:21.795431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8a8f3-47ea-4236-92b2-b5ec8ecab314 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.795731 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8a8f3-47ea-4236-92b2-b5ec8ecab314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.796001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8a8f3-47ea-4236-92b2-b5ec8ecab314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.796708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8a8f3-47ea-4236-92b2-b5ec8ecab314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:21.798354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8a8f3-47ea-4236-92b2-b5ec8ecab314 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:21.809996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1163/2339] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:21.813987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096af57-c11f-43db-8799-1ddeb33b348c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:21.824156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096af57-c11f-43db-8799-1ddeb33b348c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:21.828959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29901b10-089c-4d2d-9d3f-15c0dad69325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:21.829645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29901b10-089c-4d2d-9d3f-15c0dad69325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.829929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29901b10-089c-4d2d-9d3f-15c0dad69325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.830208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29901b10-089c-4d2d-9d3f-15c0dad69325 admin admin] PATH_INFO: `/v3/projects/e01f8e0f416b4f51a62c2d4b6e08c18a/users/5ce20723f3a74d2c89a3fcc510aeb80b/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.842678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:21.845725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29901b10-089c-4d2d-9d3f-15c0dad69325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01f8e0f416b4f51a62c2d4b6e08c18a, user_id=5ce20723f3a74d2c89a3fcc510aeb80b, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:21.849248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29901b10-089c-4d2d-9d3f-15c0dad69325 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:21.909311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8a8f3-47ea-4236-92b2-b5ec8ecab314 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:21.911917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-14a8a8f3-47ea-4236-92b2-b5ec8ecab314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:21.938468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8a8f3-47ea-4236-92b2-b5ec8ecab314 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:21.944908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:21.945634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.945925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.946207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] PATH_INFO: `/v3/projects/4b5ccae799cd4d20a9ddef31d700f625` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:21.947421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5ccae799cd4d20a9ddef31d700f625)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:21.949705 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:21.962668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1177/2340] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:21 2022] PUT /identity/v3/projects/e01f8e0f416b4f51a62c2d4b6e08c18a/users/5ce20723f3a74d2c89a3fcc510aeb80b/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:21.974969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c7a2-bef6-4598-bfba-d3288ae0a912 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:21.975269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c7a2-bef6-4598-bfba-d3288ae0a912 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:21.975601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c7a2-bef6-4598-bfba-d3288ae0a912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:22.006373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-98b4c7a2-bef6-4598-bfba-d3288ae0a912 None None] MFA Rules not processed for user `5ce20723f3a74d2c89a3fcc510aeb80b`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:22.007613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-98b4c7a2-bef6-4598-bfba-d3288ae0a912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:22.072213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1178/2341] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:22.073090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4c7a2-bef6-4598-bfba-d3288ae0a912 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:22.074385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-98b4c7a2-bef6-4598-bfba-d3288ae0a912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:22.087624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5ccae799cd4d20a9ddef31d700f625', 'request_id': 'req-72fad1c7-7173-46fd-8a77-7048670134cc'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:22.088842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:22.094562 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5ccae799cd4d20a9ddef31d700f625', 'request_id': 'req-72fad1c7-7173-46fd-8a77-7048670134cc'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:22.144666 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4c7a2-bef6-4598-bfba-d3288ae0a912 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:22.165121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5ccae799cd4d20a9ddef31d700f625', 'request_id': 'req-72fad1c7-7173-46fd-8a77-7048670134cc'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:22.181713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] The token cache is being invalidate because project 4b5ccae799cd4d20a9ddef31d700f625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:22.182097 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:22.182870 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:22.184549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1164/2342] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:21 2022] DELETE /identity/v3/projects/4b5ccae799cd4d20a9ddef31d700f625 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:22.185397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:22.186768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:22.216053 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975f854-c473-4c45-a8cd-1dfa7fa0bb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:22.217165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975f854-c473-4c45-a8cd-1dfa7fa0bb7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:22.217502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975f854-c473-4c45-a8cd-1dfa7fa0bb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:22.217805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975f854-c473-4c45-a8cd-1dfa7fa0bb7f admin admin] PATH_INFO: `/v3/users/e7ec1dc1e477434c99abb082dbf52deb` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:22.219640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975f854-c473-4c45-a8cd-1dfa7fa0bb7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ec1dc1e477434c99abb082dbf52deb)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:22.222008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975f854-c473-4c45-a8cd-1dfa7fa0bb7f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:22.268397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fad1c7-7173-46fd-8a77-7048670134cc admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:22.274552 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632452-7e45-4723-9497-ab6d01fc5e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:22.275289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632452-7e45-4723-9497-ab6d01fc5e9e admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:22.275599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632452-7e45-4723-9497-ab6d01fc5e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:22.275856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632452-7e45-4723-9497-ab6d01fc5e9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:22.276575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27632452-7e45-4723-9497-ab6d01fc5e9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:22.278686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27632452-7e45-4723-9497-ab6d01fc5e9e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:22.391542 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1165/2343] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:16:22 2022] POST /identity/v3/projects => generated 383 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:22.392654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632452-7e45-4723-9497-ab6d01fc5e9e admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:22.455322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632452-7e45-4723-9497-ab6d01fc5e9e admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:22.460718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac159a-2d40-470e-8261-6fed892e0402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:22.461377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac159a-2d40-470e-8261-6fed892e0402 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:22.463425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac159a-2d40-470e-8261-6fed892e0402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:22.463425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac159a-2d40-470e-8261-6fed892e0402 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:22.463425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac159a-2d40-470e-8261-6fed892e0402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:22.464519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac159a-2d40-470e-8261-6fed892e0402 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:22.489538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:22.502200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f975f854-c473-4c45-a8cd-1dfa7fa0bb7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ec1dc1e477434c99abb082dbf52deb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:22.502890 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f975f854-c473-4c45-a8cd-1dfa7fa0bb7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ec1dc1e477434c99abb082dbf52deb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:22.510124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f975f854-c473-4c45-a8cd-1dfa7fa0bb7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ec1dc1e477434c99abb082dbf52deb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:22.592098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1166/2344] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:16:22 2022] POST /identity/v3/users => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:22.596337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac159a-2d40-470e-8261-6fed892e0402 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:22.605927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac159a-2d40-470e-8261-6fed892e0402 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:22.610753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8de53-32f2-47ff-aba6-c155eb014fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:22.611343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8de53-32f2-47ff-aba6-c155eb014fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:22.611645 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8de53-32f2-47ff-aba6-c155eb014fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:22.611918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8de53-32f2-47ff-aba6-c155eb014fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:22.612608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8de53-32f2-47ff-aba6-c155eb014fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:22.614613 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8de53-32f2-47ff-aba6-c155eb014fd8 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:22.614736 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f975f854-c473-4c45-a8cd-1dfa7fa0bb7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ec1dc1e477434c99abb082dbf52deb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:22.615388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f975f854-c473-4c45-a8cd-1dfa7fa0bb7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ec1dc1e477434c99abb082dbf52deb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:22.627561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1167/2345] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:16:22.629537 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8de53-32f2-47ff-aba6-c155eb014fd8 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:22.631776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a6d8de53-32f2-47ff-aba6-c155eb014fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:22.636929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1179/2346] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:22 2022] DELETE /identity/v3/users/e7ec1dc1e477434c99abb082dbf52deb => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:22.641180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d789-06a1-46af-850a-58ce170a45dd None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:22.641501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d789-06a1-46af-850a-58ce170a45dd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:22.641762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d789-06a1-46af-850a-58ce170a45dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:22.645074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8de53-32f2-47ff-aba6-c155eb014fd8 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:22.660991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6e90d789-06a1-46af-850a-58ce170a45dd None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:22.662233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6e90d789-06a1-46af-850a-58ce170a45dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:22.687070 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:22.704487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc44964-1f04-434d-bf42-61175c2906b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:22.705240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc44964-1f04-434d-bf42-61175c2906b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:22.705526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc44964-1f04-434d-bf42-61175c2906b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:22.710428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc44964-1f04-434d-bf42-61175c2906b2 admin admin] PATH_INFO: `/v3/projects/9a7ca9dd814e431bb30d4a07ce385770/users/9704479db53d4c6ea6ff5c838909e065/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:22.721410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc44964-1f04-434d-bf42-61175c2906b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7ca9dd814e431bb30d4a07ce385770, user_id=9704479db53d4c6ea6ff5c838909e065, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:22.722739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1180/2347] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:16:22.724156 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc44964-1f04-434d-bf42-61175c2906b2 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:22.732735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90d789-06a1-46af-850a-58ce170a45dd None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:22.734131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6e90d789-06a1-46af-850a-58ce170a45dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:22.790083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90d789-06a1-46af-850a-58ce170a45dd None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:22.793853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e6c0c-2580-4d40-bb25-94256814cce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:22.794474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e6c0c-2580-4d40-bb25-94256814cce3 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:22.794759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e6c0c-2580-4d40-bb25-94256814cce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:22.795036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e6c0c-2580-4d40-bb25-94256814cce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:22.804599 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e6c0c-2580-4d40-bb25-94256814cce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:22.806056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e6c0c-2580-4d40-bb25-94256814cce3 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:22.815589 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1181/2348] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:16:22.855794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1168/2349] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:22 2022] PUT /identity/v3/projects/9a7ca9dd814e431bb30d4a07ce385770/users/9704479db53d4c6ea6ff5c838909e065/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:16:22.860009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e6c0c-2580-4d40-bb25-94256814cce3 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:22.871555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e6c0c-2580-4d40-bb25-94256814cce3 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:22.929160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4b395-95d4-4736-bd82-7a527f3cd2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:22.929792 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4b395-95d4-4736-bd82-7a527f3cd2db admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:22.930075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4b395-95d4-4736-bd82-7a527f3cd2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:22.930352 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4b395-95d4-4736-bd82-7a527f3cd2db admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:22.931047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4b395-95d4-4736-bd82-7a527f3cd2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:22.932651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4b395-95d4-4736-bd82-7a527f3cd2db admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:22.944418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1182/2350] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:22.947228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc44964-1f04-434d-bf42-61175c2906b2 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:22.957453 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc44964-1f04-434d-bf42-61175c2906b2 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:22.962374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac3d1e-08be-4501-9786-e9650dbc137b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:22.963023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac3d1e-08be-4501-9786-e9650dbc137b admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:22.963308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac3d1e-08be-4501-9786-e9650dbc137b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:22.963612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac3d1e-08be-4501-9786-e9650dbc137b admin admin] PATH_INFO: `/v3/projects/9a7ca9dd814e431bb30d4a07ce385770/users/9704479db53d4c6ea6ff5c838909e065/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:22.965692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac3d1e-08be-4501-9786-e9650dbc137b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7ca9dd814e431bb30d4a07ce385770, user_id=9704479db53d4c6ea6ff5c838909e065, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:22.968567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac3d1e-08be-4501-9786-e9650dbc137b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:22.978681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1169/2351] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:22 2022] PUT /identity/v3/projects/9a7ca9dd814e431bb30d4a07ce385770/users/9704479db53d4c6ea6ff5c838909e065/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:16:22.988938 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f04dd9-729d-4168-a5f6-18c508753f8d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:22.989794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f04dd9-729d-4168-a5f6-18c508753f8d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:22.990102 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f04dd9-729d-4168-a5f6-18c508753f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:22.995450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:23.021897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-17f04dd9-729d-4168-a5f6-18c508753f8d None None] MFA Rules not processed for user `9704479db53d4c6ea6ff5c838909e065`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:23.023112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-17f04dd9-729d-4168-a5f6-18c508753f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:23.098815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1183/2352] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:23.109380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac3d1e-08be-4501-9786-e9650dbc137b admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:23.111481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-40ac3d1e-08be-4501-9786-e9650dbc137b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:23.170917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac3d1e-08be-4501-9786-e9650dbc137b admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:23.174675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9dab26-bef3-4b6a-a580-a33a69357dac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:23.175297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9dab26-bef3-4b6a-a580-a33a69357dac service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:23.175582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9dab26-bef3-4b6a-a580-a33a69357dac service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:23.175858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9dab26-bef3-4b6a-a580-a33a69357dac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:23.178242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ea9dab26-bef3-4b6a-a580-a33a69357dac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:23.236880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9dab26-bef3-4b6a-a580-a33a69357dac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:23.238294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9dab26-bef3-4b6a-a580-a33a69357dac service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:23.248071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1170/2353] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:23.262181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f04dd9-729d-4168-a5f6-18c508753f8d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:23.266089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-17f04dd9-729d-4168-a5f6-18c508753f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:23.339294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f04dd9-729d-4168-a5f6-18c508753f8d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:23.345595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:23.346278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:23.346565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:23.346846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] PATH_INFO: `/v3/projects/1d81962138ca49fcb8d8e6f3776ec4b6` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:23.348103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d81962138ca49fcb8d8e6f3776ec4b6)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:23.350754 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:23.485114 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d81962138ca49fcb8d8e6f3776ec4b6', 'request_id': 'req-83e4f803-22f2-43e5-9b62-43abe14c0731'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:23.491386 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d81962138ca49fcb8d8e6f3776ec4b6', 'request_id': 'req-83e4f803-22f2-43e5-9b62-43abe14c0731'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:23.497881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d81962138ca49fcb8d8e6f3776ec4b6', 'request_id': 'req-83e4f803-22f2-43e5-9b62-43abe14c0731'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:23.597637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] The token cache is being invalidate because project 1d81962138ca49fcb8d8e6f3776ec4b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:23.597934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:23.602621 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:23.602621 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1184/2354] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:23 2022] DELETE /identity/v3/projects/1d81962138ca49fcb8d8e6f3776ec4b6 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:23.604698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9dab26-bef3-4b6a-a580-a33a69357dac service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:23.615415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9dab26-bef3-4b6a-a580-a33a69357dac service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:23.676364 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:23.677119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:23.677406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:23.677686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] PATH_INFO: `/v3/users/af43ff0dd69f40f7a151dc25992836a7` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:23.678862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af43ff0dd69f40f7a151dc25992836a7)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:23.680822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:23.694464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:23.953136 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af43ff0dd69f40f7a151dc25992836a7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:23.960574 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af43ff0dd69f40f7a151dc25992836a7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:24.084475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af43ff0dd69f40f7a151dc25992836a7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:24.085170 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af43ff0dd69f40f7a151dc25992836a7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:24.101946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af43ff0dd69f40f7a151dc25992836a7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:24.108462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1171/2355] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:23 2022] DELETE /identity/v3/users/af43ff0dd69f40f7a151dc25992836a7 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:24.531047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:24.534659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:24.540824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:24.546797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4f803-22f2-43e5-9b62-43abe14c0731 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:24.610334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:24.610984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:24.611267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:24.611559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] PATH_INFO: `/v3/projects/d9ab80831f2642719c413e13041dd6ea` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:24.612840 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ab80831f2642719c413e13041dd6ea)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:24.615309 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:24.735134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ab80831f2642719c413e13041dd6ea', 'request_id': 'req-79a2ab7f-f002-4b45-90e1-07860c5425de'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:24.741350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ab80831f2642719c413e13041dd6ea', 'request_id': 'req-79a2ab7f-f002-4b45-90e1-07860c5425de'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:24.747599 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ab80831f2642719c413e13041dd6ea', 'request_id': 'req-79a2ab7f-f002-4b45-90e1-07860c5425de'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:24.859820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] The token cache is being invalidate because project d9ab80831f2642719c413e13041dd6ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:24.860111 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:24.860812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:24.862332 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1185/2356] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:24 2022] DELETE /identity/v3/projects/d9ab80831f2642719c413e13041dd6ea => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:24.866967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:24.868397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:24.881986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc9342-ffdc-425d-9af6-3c3d28d12ad3 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:24.939367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba3950-47d3-470b-86b8-18b1a4b4e5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:24.940010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba3950-47d3-470b-86b8-18b1a4b4e5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:24.940293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba3950-47d3-470b-86b8-18b1a4b4e5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:24.940571 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba3950-47d3-470b-86b8-18b1a4b4e5d5 admin admin] PATH_INFO: `/v3/users/d6fd953955c54fad8ebe376b64c16dcd` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:24.941785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba3950-47d3-470b-86b8-18b1a4b4e5d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6fd953955c54fad8ebe376b64c16dcd)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:24.943692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba3950-47d3-470b-86b8-18b1a4b4e5d5 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:25.212581 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2aba3950-47d3-470b-86b8-18b1a4b4e5d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6fd953955c54fad8ebe376b64c16dcd'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:25.220219 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2aba3950-47d3-470b-86b8-18b1a4b4e5d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6fd953955c54fad8ebe376b64c16dcd'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:25.329443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2aba3950-47d3-470b-86b8-18b1a4b4e5d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6fd953955c54fad8ebe376b64c16dcd'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:25.330076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2aba3950-47d3-470b-86b8-18b1a4b4e5d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6fd953955c54fad8ebe376b64c16dcd'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:25.343809 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-2aba3950-47d3-470b-86b8-18b1a4b4e5d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6fd953955c54fad8ebe376b64c16dcd'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:25.350475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1172/2357] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:24 2022] DELETE /identity/v3/users/d6fd953955c54fad8ebe376b64c16dcd => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:25.782280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:25.783931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:25.799378 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2ab7f-f002-4b45-90e1-07860c5425de admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:25.856604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:25.857293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:25.857578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:25.857857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] PATH_INFO: `/v3/projects/dfdab63c1ace48ff9608722b5b695bc7` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:25.859046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfdab63c1ace48ff9608722b5b695bc7)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:25.861038 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:25.995888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:26.006843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfdab63c1ace48ff9608722b5b695bc7', 'request_id': 'req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:26.012328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfdab63c1ace48ff9608722b5b695bc7', 'request_id': 'req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:26.018585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfdab63c1ace48ff9608722b5b695bc7', 'request_id': 'req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:26.106138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] The token cache is being invalidate because project dfdab63c1ace48ff9608722b5b695bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:26.106438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:26.107069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-61d383ae-fa8f-4bad-89ff-f7271d6d5b0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:26.108596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1186/2358] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:25 2022] DELETE /identity/v3/projects/dfdab63c1ace48ff9608722b5b695bc7 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:26.115662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f78eb1-ca98-4cc7-b2e3-91155bf5382b None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:26.120762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f78eb1-ca98-4cc7-b2e3-91155bf5382b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:26.120762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f78eb1-ca98-4cc7-b2e3-91155bf5382b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:26.126089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:26.137546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-66f78eb1-ca98-4cc7-b2e3-91155bf5382b None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:26.138779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-66f78eb1-ca98-4cc7-b2e3-91155bf5382b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:26.201421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1173/2359] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:26.209347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e942777-870b-42af-8c6e-8a8f673545d8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:26.209655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e942777-870b-42af-8c6e-8a8f673545d8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:26.209917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e942777-870b-42af-8c6e-8a8f673545d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:26.232733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1e942777-870b-42af-8c6e-8a8f673545d8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:26.232733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1e942777-870b-42af-8c6e-8a8f673545d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:26.241705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1187/2360] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:26.246045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f78eb1-ca98-4cc7-b2e3-91155bf5382b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:26.259615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f78eb1-ca98-4cc7-b2e3-91155bf5382b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:26.265724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd03f9-94ae-4e0b-9f1a-dc35e22424e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:26.266634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd03f9-94ae-4e0b-9f1a-dc35e22424e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:26.266930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd03f9-94ae-4e0b-9f1a-dc35e22424e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:26.279359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd03f9-94ae-4e0b-9f1a-dc35e22424e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:26.279359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd03f9-94ae-4e0b-9f1a-dc35e22424e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:16:26.279359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd03f9-94ae-4e0b-9f1a-dc35e22424e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:26.279359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd03f9-94ae-4e0b-9f1a-dc35e22424e2 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:26.287575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1174/2361] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:16:26.292057 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e942777-870b-42af-8c6e-8a8f673545d8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:26.303573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e942777-870b-42af-8c6e-8a8f673545d8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:26.308460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b706b-26f9-454f-a008-97964a41b808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:26.311957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b706b-26f9-454f-a008-97964a41b808 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:26.312367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b706b-26f9-454f-a008-97964a41b808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:26.312651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b706b-26f9-454f-a008-97964a41b808 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:26.313559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b706b-26f9-454f-a008-97964a41b808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:26.315510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b706b-26f9-454f-a008-97964a41b808 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:26.391068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1188/2362] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:16:26 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:26.395352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd03f9-94ae-4e0b-9f1a-dc35e22424e2 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:26.405711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd03f9-94ae-4e0b-9f1a-dc35e22424e2 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:26.481010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055c7be-bd9d-4660-9ee7-08c178a375ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:26.481801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055c7be-bd9d-4660-9ee7-08c178a375ae admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:26.482181 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055c7be-bd9d-4660-9ee7-08c178a375ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:26.482593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055c7be-bd9d-4660-9ee7-08c178a375ae admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:26.483276 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055c7be-bd9d-4660-9ee7-08c178a375ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:26.486046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055c7be-bd9d-4660-9ee7-08c178a375ae admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:26.586954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1175/2363] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:16:26 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:26.591127 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b706b-26f9-454f-a008-97964a41b808 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:26.602364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b706b-26f9-454f-a008-97964a41b808 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:26.607120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a135629-7716-4bbe-ad47-6907a4274360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:26.607743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135629-7716-4bbe-ad47-6907a4274360 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:26.608028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135629-7716-4bbe-ad47-6907a4274360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:26.608306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135629-7716-4bbe-ad47-6907a4274360 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:26.609041 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a135629-7716-4bbe-ad47-6907a4274360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:26.610639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a135629-7716-4bbe-ad47-6907a4274360 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:26.624285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1189/2364] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:26.625533 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055c7be-bd9d-4660-9ee7-08c178a375ae admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:26.635873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055c7be-bd9d-4660-9ee7-08c178a375ae admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:26.640609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673fbb5-cba9-4233-a6b3-e40d6bdc68b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:26.641264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673fbb5-cba9-4233-a6b3-e40d6bdc68b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:26.641550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673fbb5-cba9-4233-a6b3-e40d6bdc68b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:26.641827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673fbb5-cba9-4233-a6b3-e40d6bdc68b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:26.642527 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673fbb5-cba9-4233-a6b3-e40d6bdc68b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:26.644114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673fbb5-cba9-4233-a6b3-e40d6bdc68b9 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:26.655585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1176/2365] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:26.660183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a135629-7716-4bbe-ad47-6907a4274360 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:26.671197 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a135629-7716-4bbe-ad47-6907a4274360 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:26.675867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787649df-a4c9-4415-bdf9-d039b0c361a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:26.676539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787649df-a4c9-4415-bdf9-d039b0c361a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:26.676823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787649df-a4c9-4415-bdf9-d039b0c361a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:26.677142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787649df-a4c9-4415-bdf9-d039b0c361a4 admin admin] PATH_INFO: `/v3/projects/308e6cf086ae4cdea9c591ed2811c976/users/30f0002d2a5a4950b2f1aa771f33bbc1/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:26.691411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787649df-a4c9-4415-bdf9-d039b0c361a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=308e6cf086ae4cdea9c591ed2811c976, user_id=30f0002d2a5a4950b2f1aa771f33bbc1, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:26.694754 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787649df-a4c9-4415-bdf9-d039b0c361a4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:26.809696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1190/2366] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:26 2022] PUT /identity/v3/projects/308e6cf086ae4cdea9c591ed2811c976/users/30f0002d2a5a4950b2f1aa771f33bbc1/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:26.814744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780baaa-79b2-411c-be86-ee1cdc04bd5f None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:26.815032 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780baaa-79b2-411c-be86-ee1cdc04bd5f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:26.815301 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780baaa-79b2-411c-be86-ee1cdc04bd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:26.835740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0780baaa-79b2-411c-be86-ee1cdc04bd5f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:26.836905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0780baaa-79b2-411c-be86-ee1cdc04bd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:26.899638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1177/2367] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:26.909519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787649df-a4c9-4415-bdf9-d039b0c361a4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:26.911887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-787649df-a4c9-4415-bdf9-d039b0c361a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:26.986232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787649df-a4c9-4415-bdf9-d039b0c361a4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:26.990213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf5177-2cee-476b-926e-dc85d1de0cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:26.990843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf5177-2cee-476b-926e-dc85d1de0cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:26.991131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf5177-2cee-476b-926e-dc85d1de0cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:26.991410 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf5177-2cee-476b-926e-dc85d1de0cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:27.000699 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf5177-2cee-476b-926e-dc85d1de0cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:27.002235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf5177-2cee-476b-926e-dc85d1de0cc8 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:27.014094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1191/2368] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:31.978718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780baaa-79b2-411c-be86-ee1cdc04bd5f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:31.980997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0780baaa-79b2-411c-be86-ee1cdc04bd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:31.996473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:32.047399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780baaa-79b2-411c-be86-ee1cdc04bd5f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:32.052188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f3d87-349f-4277-840b-60108b53cb9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:32.052901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f3d87-349f-4277-840b-60108b53cb9c service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:32.053057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f3d87-349f-4277-840b-60108b53cb9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:32.053331 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f3d87-349f-4277-840b-60108b53cb9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:32.056405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c26f3d87-349f-4277-840b-60108b53cb9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:32.135552 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f3d87-349f-4277-840b-60108b53cb9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:32.137171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f3d87-349f-4277-840b-60108b53cb9c service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:32.148990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1178/2369] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:32.639764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b79d6f-ee21-465b-86ae-82511b92fbe4 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:32.639764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b79d6f-ee21-465b-86ae-82511b92fbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:32.639764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b79d6f-ee21-465b-86ae-82511b92fbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:32.681368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-12b79d6f-ee21-465b-86ae-82511b92fbe4 None None] MFA Rules not processed for user `30f0002d2a5a4950b2f1aa771f33bbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:32.682670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-12b79d6f-ee21-465b-86ae-82511b92fbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:32.767514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:32.778405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1192/2370] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:32.783159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f3d87-349f-4277-840b-60108b53cb9c service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:32.797618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f3d87-349f-4277-840b-60108b53cb9c service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:32.803332 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da20d3-1f32-4b31-82c3-492de205b93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:32.804081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da20d3-1f32-4b31-82c3-492de205b93a admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:32.804446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da20d3-1f32-4b31-82c3-492de205b93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:32.804806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da20d3-1f32-4b31-82c3-492de205b93a admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:32.805620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da20d3-1f32-4b31-82c3-492de205b93a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:32.807614 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da20d3-1f32-4b31-82c3-492de205b93a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:32.908847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1179/2371] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:16:32 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:16:32.913183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b79d6f-ee21-465b-86ae-82511b92fbe4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:32.927780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b79d6f-ee21-465b-86ae-82511b92fbe4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:32.998439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6863f39-77a4-4055-977c-a93e4a8ff6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:32.999392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6863f39-77a4-4055-977c-a93e4a8ff6c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:32.999872 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6863f39-77a4-4055-977c-a93e4a8ff6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:33.000361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6863f39-77a4-4055-977c-a93e4a8ff6c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:33.001328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6863f39-77a4-4055-977c-a93e4a8ff6c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:33.003728 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6863f39-77a4-4055-977c-a93e4a8ff6c0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:33.110400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1193/2372] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:16:32 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:16:33.112251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da20d3-1f32-4b31-82c3-492de205b93a admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:33.124302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da20d3-1f32-4b31-82c3-492de205b93a admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:33.129110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124fe92-212a-4af4-ac1d-a1569bfbb38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:33.133001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124fe92-212a-4af4-ac1d-a1569bfbb38f admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:33.133001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124fe92-212a-4af4-ac1d-a1569bfbb38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:33.133001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124fe92-212a-4af4-ac1d-a1569bfbb38f admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:33.133001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124fe92-212a-4af4-ac1d-a1569bfbb38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:33.133001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124fe92-212a-4af4-ac1d-a1569bfbb38f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:33.146371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1180/2373] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:33.150674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6863f39-77a4-4055-977c-a93e4a8ff6c0 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:33.162125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6863f39-77a4-4055-977c-a93e4a8ff6c0 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:33.171573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c71f8-f904-4d54-8c90-39147f4c109f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:33.172596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c71f8-f904-4d54-8c90-39147f4c109f admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:33.172905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c71f8-f904-4d54-8c90-39147f4c109f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:33.173227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c71f8-f904-4d54-8c90-39147f4c109f admin admin] PATH_INFO: `/v3/projects/7694d36e7d80422ab3f1d49efc71ec10/users/a0719f10bb0c4a30b373ee6e2be69d93/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:33.190853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c71f8-f904-4d54-8c90-39147f4c109f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7694d36e7d80422ab3f1d49efc71ec10, user_id=a0719f10bb0c4a30b373ee6e2be69d93, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:33.193644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c71f8-f904-4d54-8c90-39147f4c109f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:33.269591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1194/2374] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:33 2022] PUT /identity/v3/projects/7694d36e7d80422ab3f1d49efc71ec10/users/a0719f10bb0c4a30b373ee6e2be69d93/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:33.273642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124fe92-212a-4af4-ac1d-a1569bfbb38f admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:33.287246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124fe92-212a-4af4-ac1d-a1569bfbb38f admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:33.350331 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e52f5-cdca-47c7-9abc-46f911b3e622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:33.350975 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e52f5-cdca-47c7-9abc-46f911b3e622 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:33.351293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e52f5-cdca-47c7-9abc-46f911b3e622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:33.351593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e52f5-cdca-47c7-9abc-46f911b3e622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:33.352307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e52f5-cdca-47c7-9abc-46f911b3e622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:33.353964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e52f5-cdca-47c7-9abc-46f911b3e622 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:33.365554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1181/2375] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:33.369807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c71f8-f904-4d54-8c90-39147f4c109f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:33.381055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c71f8-f904-4d54-8c90-39147f4c109f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:33.385853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87eca8a-5048-425a-864a-4bc08f1a36d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:33.386532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eca8a-5048-425a-864a-4bc08f1a36d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:33.386817 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eca8a-5048-425a-864a-4bc08f1a36d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:33.387093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eca8a-5048-425a-864a-4bc08f1a36d3 admin admin] PATH_INFO: `/v3/projects/7694d36e7d80422ab3f1d49efc71ec10/users/a0719f10bb0c4a30b373ee6e2be69d93/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:33.389069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87eca8a-5048-425a-864a-4bc08f1a36d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7694d36e7d80422ab3f1d49efc71ec10, user_id=a0719f10bb0c4a30b373ee6e2be69d93, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:33.391768 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87eca8a-5048-425a-864a-4bc08f1a36d3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:33.403381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1195/2376] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:33 2022] PUT /identity/v3/projects/7694d36e7d80422ab3f1d49efc71ec10/users/a0719f10bb0c4a30b373ee6e2be69d93/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:39.606373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109c0dc-7305-42b2-bc0b-4358149c6869 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:39.606883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109c0dc-7305-42b2-bc0b-4358149c6869 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:39.606966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109c0dc-7305-42b2-bc0b-4358149c6869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:39.648913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e109c0dc-7305-42b2-bc0b-4358149c6869 None None] MFA Rules not processed for user `a0719f10bb0c4a30b373ee6e2be69d93`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:39.650227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e109c0dc-7305-42b2-bc0b-4358149c6869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:39.704354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:39.747386 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87eca8a-5048-425a-864a-4bc08f1a36d3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:39.749453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f87eca8a-5048-425a-864a-4bc08f1a36d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:39.759409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1182/2377] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:39.768139 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109c0dc-7305-42b2-bc0b-4358149c6869 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:39.772711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e109c0dc-7305-42b2-bc0b-4358149c6869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:39.839098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87eca8a-5048-425a-864a-4bc08f1a36d3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:39.845600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:39.846550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:39.846836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:39.847110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] PATH_INFO: `/v3/users/5ce20723f3a74d2c89a3fcc510aeb80b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:39.849533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce20723f3a74d2c89a3fcc510aeb80b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:39.851726 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:39.865872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109c0dc-7305-42b2-bc0b-4358149c6869 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:39.872426 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c638f-b1bd-4ba0-8342-2e0f87579d59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:39.873168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c638f-b1bd-4ba0-8342-2e0f87579d59 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:39.873523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c638f-b1bd-4ba0-8342-2e0f87579d59 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:39.873877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c638f-b1bd-4ba0-8342-2e0f87579d59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:39.937053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c638f-b1bd-4ba0-8342-2e0f87579d59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:39.938722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c638f-b1bd-4ba0-8342-2e0f87579d59 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:39.948557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1183/2378] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:40.186913 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:40.190695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce20723f3a74d2c89a3fcc510aeb80b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:40.190695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce20723f3a74d2c89a3fcc510aeb80b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:40.212080 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce20723f3a74d2c89a3fcc510aeb80b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:40.294830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce20723f3a74d2c89a3fcc510aeb80b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:40.295488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce20723f3a74d2c89a3fcc510aeb80b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:40.327032 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1196/2379] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:39 2022] DELETE /identity/v3/users/5ce20723f3a74d2c89a3fcc510aeb80b => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:40.334853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b3f7f-1bfe-4920-815a-969a3dc3b7ce None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:40.334853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b3f7f-1bfe-4920-815a-969a3dc3b7ce None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:40.334853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b3f7f-1bfe-4920-815a-969a3dc3b7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:40.354397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1b4b3f7f-1bfe-4920-815a-969a3dc3b7ce None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:40.355546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1b4b3f7f-1bfe-4920-815a-969a3dc3b7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:40.401448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:40.421347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1184/2380] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:40.431993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:40.434895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:40.500893 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e926b-4994-4cd5-883f-ed05e7044770 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:40.505725 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311ecd9-b139-44f2-8c63-ff6c2136fcfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:40.506791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311ecd9-b139-44f2-8c63-ff6c2136fcfe service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:40.506914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311ecd9-b139-44f2-8c63-ff6c2136fcfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:40.507200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311ecd9-b139-44f2-8c63-ff6c2136fcfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:40.517970 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311ecd9-b139-44f2-8c63-ff6c2136fcfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:40.519564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311ecd9-b139-44f2-8c63-ff6c2136fcfe service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:40.531074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1197/2381] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:41.016674 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b3f7f-1bfe-4920-815a-969a3dc3b7ce None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:41.018215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1b4b3f7f-1bfe-4920-815a-969a3dc3b7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:41.035150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b3f7f-1bfe-4920-815a-969a3dc3b7ce None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:41.042739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:41.043931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:41.044406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:41.044842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] PATH_INFO: `/v3/projects/e01f8e0f416b4f51a62c2d4b6e08c18a` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:41.047349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01f8e0f416b4f51a62c2d4b6e08c18a)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:41.049628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:41.158959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01f8e0f416b4f51a62c2d4b6e08c18a', 'request_id': 'req-72a0ca68-645e-4881-95db-cd76bcd4ac3b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:41.165370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01f8e0f416b4f51a62c2d4b6e08c18a', 'request_id': 'req-72a0ca68-645e-4881-95db-cd76bcd4ac3b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:41.242494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01f8e0f416b4f51a62c2d4b6e08c18a', 'request_id': 'req-72a0ca68-645e-4881-95db-cd76bcd4ac3b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:41.259106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] The token cache is being invalidate because project e01f8e0f416b4f51a62c2d4b6e08c18a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:41.259472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:41.260240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:41.262198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1185/2382] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:41 2022] DELETE /identity/v3/projects/e01f8e0f416b4f51a62c2d4b6e08c18a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:41.266075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311ecd9-b139-44f2-8c63-ff6c2136fcfe service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:41.277417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311ecd9-b139-44f2-8c63-ff6c2136fcfe service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:41.348129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a3ba3-68cb-4ac7-8e73-8d73e4a7a5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:41.349347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a3ba3-68cb-4ac7-8e73-8d73e4a7a5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:41.349625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a3ba3-68cb-4ac7-8e73-8d73e4a7a5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:41.349894 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a3ba3-68cb-4ac7-8e73-8d73e4a7a5c3 admin admin] PATH_INFO: `/v3/users/9704479db53d4c6ea6ff5c838909e065` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:41.351076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a3ba3-68cb-4ac7-8e73-8d73e4a7a5c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9704479db53d4c6ea6ff5c838909e065)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:41.353098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a3ba3-68cb-4ac7-8e73-8d73e4a7a5c3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:41.492219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fd7a3ba3-68cb-4ac7-8e73-8d73e4a7a5c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9704479db53d4c6ea6ff5c838909e065'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:41.492956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fd7a3ba3-68cb-4ac7-8e73-8d73e4a7a5c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9704479db53d4c6ea6ff5c838909e065'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:41.500366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fd7a3ba3-68cb-4ac7-8e73-8d73e4a7a5c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9704479db53d4c6ea6ff5c838909e065'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:41.562907 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fd7a3ba3-68cb-4ac7-8e73-8d73e4a7a5c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9704479db53d4c6ea6ff5c838909e065'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:41.563647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fd7a3ba3-68cb-4ac7-8e73-8d73e4a7a5c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9704479db53d4c6ea6ff5c838909e065'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:41.572039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:41.586820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1198/2383] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:41 2022] DELETE /identity/v3/users/9704479db53d4c6ea6ff5c838909e065 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:16:42.131705 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:42.133342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:42.147190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0ca68-645e-4881-95db-cd76bcd4ac3b admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:42.210098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:42.210836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:42.211195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:42.211470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] PATH_INFO: `/v3/projects/9a7ca9dd814e431bb30d4a07ce385770` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:42.212907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a7ca9dd814e431bb30d4a07ce385770)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:42.214865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:42.336600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a7ca9dd814e431bb30d4a07ce385770', 'request_id': 'req-586b73cf-bbfa-4240-bafb-cbed4fd779ed'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:42.342964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a7ca9dd814e431bb30d4a07ce385770', 'request_id': 'req-586b73cf-bbfa-4240-bafb-cbed4fd779ed'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:42.443738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a7ca9dd814e431bb30d4a07ce385770', 'request_id': 'req-586b73cf-bbfa-4240-bafb-cbed4fd779ed'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:42.452431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:42.456432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] The token cache is being invalidate because project 9a7ca9dd814e431bb30d4a07ce385770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:42.456722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:42.457378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-586b73cf-bbfa-4240-bafb-cbed4fd779ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:42.458923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1186/2384] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:42 2022] DELETE /identity/v3/projects/9a7ca9dd814e431bb30d4a07ce385770 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:42.464772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dbd9b-c95b-4294-a60e-c859198a9d7f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:42.465286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dbd9b-c95b-4294-a60e-c859198a9d7f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:42.465432 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dbd9b-c95b-4294-a60e-c859198a9d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:42.485841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1f2dbd9b-c95b-4294-a60e-c859198a9d7f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:42.487174 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1f2dbd9b-c95b-4294-a60e-c859198a9d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:42.549499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1199/2385] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:42.559425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863c3c8-4202-429a-9b4b-83fdd646edbd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:42.559708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863c3c8-4202-429a-9b4b-83fdd646edbd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:42.559982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863c3c8-4202-429a-9b4b-83fdd646edbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:42.579943 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a863c3c8-4202-429a-9b4b-83fdd646edbd None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:42.581180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a863c3c8-4202-429a-9b4b-83fdd646edbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:42.589968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1187/2386] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:42.594398 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2dbd9b-c95b-4294-a60e-c859198a9d7f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:42.604020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2dbd9b-c95b-4294-a60e-c859198a9d7f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:42.608698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558354c9-b662-450b-bc84-5decedac0ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:42.609348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558354c9-b662-450b-bc84-5decedac0ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:42.609630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558354c9-b662-450b-bc84-5decedac0ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:42.609904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558354c9-b662-450b-bc84-5decedac0ea8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:42.610851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558354c9-b662-450b-bc84-5decedac0ea8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:16:42.611157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558354c9-b662-450b-bc84-5decedac0ea8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:42.612744 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558354c9-b662-450b-bc84-5decedac0ea8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:42.628672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1200/2387] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:16:42.629905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863c3c8-4202-429a-9b4b-83fdd646edbd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:42.640527 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863c3c8-4202-429a-9b4b-83fdd646edbd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:42.646008 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25fcd8-2213-4afb-9776-d2165d21d547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:42.646670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25fcd8-2213-4afb-9776-d2165d21d547 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:42.646956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25fcd8-2213-4afb-9776-d2165d21d547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:42.647233 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25fcd8-2213-4afb-9776-d2165d21d547 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:42.647917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25fcd8-2213-4afb-9776-d2165d21d547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:42.649910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25fcd8-2213-4afb-9776-d2165d21d547 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:42.748898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1188/2388] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:16:42 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:42.752867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558354c9-b662-450b-bc84-5decedac0ea8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:42.765583 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558354c9-b662-450b-bc84-5decedac0ea8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:42.831552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dedd3-6900-4935-b8f9-332efc19bda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:42.832180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dedd3-6900-4935-b8f9-332efc19bda6 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:42.832464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dedd3-6900-4935-b8f9-332efc19bda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:42.832741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dedd3-6900-4935-b8f9-332efc19bda6 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:42.833450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8dedd3-6900-4935-b8f9-332efc19bda6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:42.835390 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8dedd3-6900-4935-b8f9-332efc19bda6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:42.934731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1201/2389] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:16:42 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:42.938856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25fcd8-2213-4afb-9776-d2165d21d547 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:42.949476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25fcd8-2213-4afb-9776-d2165d21d547 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:42.954109 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cea5c-7e0e-41c3-bb9e-a2372f33be97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:42.954716 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cea5c-7e0e-41c3-bb9e-a2372f33be97 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:42.955004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cea5c-7e0e-41c3-bb9e-a2372f33be97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:42.955308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cea5c-7e0e-41c3-bb9e-a2372f33be97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:42.956044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670cea5c-7e0e-41c3-bb9e-a2372f33be97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:42.957753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670cea5c-7e0e-41c3-bb9e-a2372f33be97 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:42.968000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1189/2390] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:42.972586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dedd3-6900-4935-b8f9-332efc19bda6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:42.995943 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dedd3-6900-4935-b8f9-332efc19bda6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:43.000870 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67725367-74bc-42c5-9c57-0139c8b16637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:43.001534 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67725367-74bc-42c5-9c57-0139c8b16637 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:43.001818 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67725367-74bc-42c5-9c57-0139c8b16637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:43.002096 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67725367-74bc-42c5-9c57-0139c8b16637 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:43.002806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67725367-74bc-42c5-9c57-0139c8b16637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:43.004407 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67725367-74bc-42c5-9c57-0139c8b16637 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:43.016143 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1202/2391] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:43.020269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cea5c-7e0e-41c3-bb9e-a2372f33be97 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:43.031865 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cea5c-7e0e-41c3-bb9e-a2372f33be97 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:43.036800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62726e11-2ad7-44e8-96a7-e8990a6ba6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:43.038190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62726e11-2ad7-44e8-96a7-e8990a6ba6b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:43.038464 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62726e11-2ad7-44e8-96a7-e8990a6ba6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:43.038745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62726e11-2ad7-44e8-96a7-e8990a6ba6b1 admin admin] PATH_INFO: `/v3/projects/9365382647b14c85bea14c501fc09c0c/users/79d5d840d15440fa990a124b3941ca05/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:43.055172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62726e11-2ad7-44e8-96a7-e8990a6ba6b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9365382647b14c85bea14c501fc09c0c, user_id=79d5d840d15440fa990a124b3941ca05, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:43.057937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62726e11-2ad7-44e8-96a7-e8990a6ba6b1 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:43.135432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1190/2392] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:43 2022] PUT /identity/v3/projects/9365382647b14c85bea14c501fc09c0c/users/79d5d840d15440fa990a124b3941ca05/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:43.213238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4df6-a0f4-4d12-9756-27a6a85e4fb0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:43.213785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4df6-a0f4-4d12-9756-27a6a85e4fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:43.214433 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4df6-a0f4-4d12-9756-27a6a85e4fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:43.250015 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c8fb4df6-a0f4-4d12-9756-27a6a85e4fb0 None None] MFA Rules not processed for user `79d5d840d15440fa990a124b3941ca05`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:43.251321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c8fb4df6-a0f4-4d12-9756-27a6a85e4fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:43.299738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:43.329106 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1203/2393] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:16:43.341074 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62726e11-2ad7-44e8-96a7-e8990a6ba6b1 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:43.343147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-62726e11-2ad7-44e8-96a7-e8990a6ba6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:43.408857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62726e11-2ad7-44e8-96a7-e8990a6ba6b1 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:43.413092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4a4cb-c489-4152-8822-16091145aa5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:43.413733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4a4cb-c489-4152-8822-16091145aa5c service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:43.414012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4a4cb-c489-4152-8822-16091145aa5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:43.414299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4a4cb-c489-4152-8822-16091145aa5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:43.424687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4a4cb-c489-4152-8822-16091145aa5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:43.426187 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4a4cb-c489-4152-8822-16091145aa5c service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:43.436322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1191/2394] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:43.466092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb4df6-a0f4-4d12-9756-27a6a85e4fb0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:43.548869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb4df6-a0f4-4d12-9756-27a6a85e4fb0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:43.554698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:43.556678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:43.557070 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:43.557422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] PATH_INFO: `/v3/users/79d5d840d15440fa990a124b3941ca05` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:43.559877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79d5d840d15440fa990a124b3941ca05)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:43.562669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:43.845994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79d5d840d15440fa990a124b3941ca05'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:43.846961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79d5d840d15440fa990a124b3941ca05'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:43.855257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79d5d840d15440fa990a124b3941ca05'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:43.991452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79d5d840d15440fa990a124b3941ca05'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:43.992103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79d5d840d15440fa990a124b3941ca05'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:44.010400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1204/2395] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:16:43 2022] DELETE /identity/v3/users/79d5d840d15440fa990a124b3941ca05 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:44.014889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422236b-1198-4007-9193-3eb442a189de None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:44.015226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422236b-1198-4007-9193-3eb442a189de None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:44.015480 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422236b-1198-4007-9193-3eb442a189de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:44.035593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-b422236b-1198-4007-9193-3eb442a189de None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:44.037400 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b422236b-1198-4007-9193-3eb442a189de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:44.102814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1192/2396] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:44.112722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:44.115630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:44.206102 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c4816-9963-426c-b614-a2b9ca4898b4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:44.225043 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018163f2-6a4c-4cae-b1e3-87f40259fcee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:44.226639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018163f2-6a4c-4cae-b1e3-87f40259fcee service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:44.226984 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018163f2-6a4c-4cae-b1e3-87f40259fcee service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:44.227731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018163f2-6a4c-4cae-b1e3-87f40259fcee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:44.246011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018163f2-6a4c-4cae-b1e3-87f40259fcee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:44.247982 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018163f2-6a4c-4cae-b1e3-87f40259fcee service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:44.269434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1205/2397] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:44.800370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422236b-1198-4007-9193-3eb442a189de None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:44.801726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b422236b-1198-4007-9193-3eb442a189de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:44.816356 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422236b-1198-4007-9193-3eb442a189de None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:44.822840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:44.823836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:44.824242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:44.824589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] PATH_INFO: `/v3/projects/9365382647b14c85bea14c501fc09c0c` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:44.825972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9365382647b14c85bea14c501fc09c0c)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:44.828072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:45.019109 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:45.039178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9365382647b14c85bea14c501fc09c0c', 'request_id': 'req-c03861ed-0b29-4335-8b64-72e36d733b03'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:45.055903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9365382647b14c85bea14c501fc09c0c', 'request_id': 'req-c03861ed-0b29-4335-8b64-72e36d733b03'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:45.174820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9365382647b14c85bea14c501fc09c0c', 'request_id': 'req-c03861ed-0b29-4335-8b64-72e36d733b03'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:45.191664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] The token cache is being invalidate because project 9365382647b14c85bea14c501fc09c0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:16:45.192025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:45.192704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c03861ed-0b29-4335-8b64-72e36d733b03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:16:45.197159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1193/2398] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:16:44 2022] DELETE /identity/v3/projects/9365382647b14c85bea14c501fc09c0c => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:45.202565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a8324-c36d-4bc0-ac20-c2caea984c3f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:45.203062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a8324-c36d-4bc0-ac20-c2caea984c3f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:45.203169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a8324-c36d-4bc0-ac20-c2caea984c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:45.218888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:45.232788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e50a8324-c36d-4bc0-ac20-c2caea984c3f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:45.234188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e50a8324-c36d-4bc0-ac20-c2caea984c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:45.314478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1206/2399] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:45.323027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744096d8-13d9-4339-ad37-0a3e0473db34 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:45.323429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744096d8-13d9-4339-ad37-0a3e0473db34 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:45.323711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744096d8-13d9-4339-ad37-0a3e0473db34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:45.348539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-744096d8-13d9-4339-ad37-0a3e0473db34 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:45.350565 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-744096d8-13d9-4339-ad37-0a3e0473db34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:45.359559 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1194/2400] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:45.363996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a8324-c36d-4bc0-ac20-c2caea984c3f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:45.374513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a8324-c36d-4bc0-ac20-c2caea984c3f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:45.379401 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762c771-f558-4277-ac45-ebf4020c8dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:45.380559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762c771-f558-4277-ac45-ebf4020c8dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:45.380559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762c771-f558-4277-ac45-ebf4020c8dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:45.380923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762c771-f558-4277-ac45-ebf4020c8dcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:45.381675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762c771-f558-4277-ac45-ebf4020c8dcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:16:45.381972 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762c771-f558-4277-ac45-ebf4020c8dcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:45.383566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762c771-f558-4277-ac45-ebf4020c8dcb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:45.397225 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1207/2401] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:16:45.401764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744096d8-13d9-4339-ad37-0a3e0473db34 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:45.411887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744096d8-13d9-4339-ad37-0a3e0473db34 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:45.417744 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84e920-9c27-4871-8fa0-a98529c773ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:45.418381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84e920-9c27-4871-8fa0-a98529c773ac admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:45.418668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84e920-9c27-4871-8fa0-a98529c773ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:45.418945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84e920-9c27-4871-8fa0-a98529c773ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:45.419632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84e920-9c27-4871-8fa0-a98529c773ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:45.421804 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84e920-9c27-4871-8fa0-a98529c773ac admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:45.535722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1195/2402] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:16:45 2022] POST /identity/v3/projects => generated 369 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:45.540050 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762c771-f558-4277-ac45-ebf4020c8dcb admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:45.550355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762c771-f558-4277-ac45-ebf4020c8dcb admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:45.609600 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c82e3b-2f6f-4036-8243-f417dc21d2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:45.610219 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c82e3b-2f6f-4036-8243-f417dc21d2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:45.610500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c82e3b-2f6f-4036-8243-f417dc21d2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:45.610779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c82e3b-2f6f-4036-8243-f417dc21d2b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:45.611453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c82e3b-2f6f-4036-8243-f417dc21d2b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:45.613541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c82e3b-2f6f-4036-8243-f417dc21d2b9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:45.726244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1208/2403] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:16:45 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:16:45.730653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84e920-9c27-4871-8fa0-a98529c773ac admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:45.741720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84e920-9c27-4871-8fa0-a98529c773ac admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:45.746659 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c8636-973e-4224-af6f-ed082957a780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:45.747272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c8636-973e-4224-af6f-ed082957a780 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:45.747551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c8636-973e-4224-af6f-ed082957a780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:45.747822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c8636-973e-4224-af6f-ed082957a780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:45.748521 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c8636-973e-4224-af6f-ed082957a780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:45.750540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c8636-973e-4224-af6f-ed082957a780 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:45.761946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1196/2404] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:45.766163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c82e3b-2f6f-4036-8243-f417dc21d2b9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:45.777371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c82e3b-2f6f-4036-8243-f417dc21d2b9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:45.796079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360b5f8-8514-43f3-9f05-da6e57672e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:45.796079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360b5f8-8514-43f3-9f05-da6e57672e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:45.796079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360b5f8-8514-43f3-9f05-da6e57672e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:45.796079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360b5f8-8514-43f3-9f05-da6e57672e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:45.796572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360b5f8-8514-43f3-9f05-da6e57672e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:45.796572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360b5f8-8514-43f3-9f05-da6e57672e8a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:45.803287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1209/2405] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:16:45.807469 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c8636-973e-4224-af6f-ed082957a780 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:45.819933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c8636-973e-4224-af6f-ed082957a780 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:45.825315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ccd1-029b-41d6-8400-281f97b21414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:45.830890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ccd1-029b-41d6-8400-281f97b21414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:45.830890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ccd1-029b-41d6-8400-281f97b21414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:45.830890 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ccd1-029b-41d6-8400-281f97b21414 admin admin] PATH_INFO: `/v3/projects/a55e72ee6d5f4525ad37b9fb366c7506/users/bd62d8053b544f8bb1ca173cd007dbef/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:45.842640 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27ccd1-029b-41d6-8400-281f97b21414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a55e72ee6d5f4525ad37b9fb366c7506, user_id=bd62d8053b544f8bb1ca173cd007dbef, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:45.845350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27ccd1-029b-41d6-8400-281f97b21414 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:45.945856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1197/2406] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:16:45 2022] PUT /identity/v3/projects/a55e72ee6d5f4525ad37b9fb366c7506/users/bd62d8053b544f8bb1ca173cd007dbef/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:16:45.957479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d79283-0cdc-4292-a076-d81a96b375fc None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:45.957760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d79283-0cdc-4292-a076-d81a96b375fc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:45.958089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d79283-0cdc-4292-a076-d81a96b375fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:45.996420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-04d79283-0cdc-4292-a076-d81a96b375fc None None] MFA Rules not processed for user `bd62d8053b544f8bb1ca173cd007dbef`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:16:45.997779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-04d79283-0cdc-4292-a076-d81a96b375fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:46.084688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1210/2407] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:16:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:16:46.096721 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ccd1-029b-41d6-8400-281f97b21414 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:16:46.098496 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0a27ccd1-029b-41d6-8400-281f97b21414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:16:46.145993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:46.160509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ccd1-029b-41d6-8400-281f97b21414 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:16:46.164538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41c565-d5a0-4c74-8e5c-b0c339c442cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47f1552e73648ceb8a421c0543c2974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:16:46.165241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41c565-d5a0-4c74-8e5c-b0c339c442cf service glance] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:16:46.165548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41c565-d5a0-4c74-8e5c-b0c339c442cf service glance] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:16:46.165844 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41c565-d5a0-4c74-8e5c-b0c339c442cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:16:46.181567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41c565-d5a0-4c74-8e5c-b0c339c442cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:16:46.181567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41c565-d5a0-4c74-8e5c-b0c339c442cf service glance] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:16:46.187016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1198/2408] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:16:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:16:56.182790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:16:56.225987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:17:00.412528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d79283-0cdc-4292-a076-d81a96b375fc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:00.415344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-04d79283-0cdc-4292-a076-d81a96b375fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:00.508006 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d79283-0cdc-4292-a076-d81a96b375fc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:00.513268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:00.513939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:00.514230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:00.514495 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] PATH_INFO: `/v3/users/7106f76d179e4775b0a8e7e3b819df9d` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:00.515781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7106f76d179e4775b0a8e7e3b819df9d)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:00.517835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:00.679845 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7106f76d179e4775b0a8e7e3b819df9d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:17:00.680412 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7106f76d179e4775b0a8e7e3b819df9d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:17:00.687402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7106f76d179e4775b0a8e7e3b819df9d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:17:00.745751 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7106f76d179e4775b0a8e7e3b819df9d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:17:00.746375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7106f76d179e4775b0a8e7e3b819df9d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:17:00.766602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1211/2409] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:17:00 2022] DELETE /identity/v3/users/7106f76d179e4775b0a8e7e3b819df9d => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:17:00.772769 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efed81-8802-49a0-8d79-9e8ae4111176 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:00.773391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efed81-8802-49a0-8d79-9e8ae4111176 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:00.773944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efed81-8802-49a0-8d79-9e8ae4111176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:00.797168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f4efed81-8802-49a0-8d79-9e8ae4111176 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:00.798557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f4efed81-8802-49a0-8d79-9e8ae4111176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:00.858701 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1199/2410] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:17:00.868214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:00.869653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:00.940946 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e159d-1b85-440c-99ea-236094e49138 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:00.945119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00e7e0-0fb5-4441-b130-2dcbf1932f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:00.945743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00e7e0-0fb5-4441-b130-2dcbf1932f7d service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:00.946032 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00e7e0-0fb5-4441-b130-2dcbf1932f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:00.946307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00e7e0-0fb5-4441-b130-2dcbf1932f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:00.956554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00e7e0-0fb5-4441-b130-2dcbf1932f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:00.958104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00e7e0-0fb5-4441-b130-2dcbf1932f7d service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:00.969423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1212/2411] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:17:01.675275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efed81-8802-49a0-8d79-9e8ae4111176 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:01.676022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f4efed81-8802-49a0-8d79-9e8ae4111176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:01.696543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efed81-8802-49a0-8d79-9e8ae4111176 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:01.701585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:01.702507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:01.702507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:01.702913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] PATH_INFO: `/v3/projects/10251a1b077c4066a1898bdab6bb37dd` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:01.704044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=10251a1b077c4066a1898bdab6bb37dd)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:01.706003 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:01.839185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10251a1b077c4066a1898bdab6bb37dd', 'request_id': 'req-1a79ce75-f244-432f-b57b-a0b95a13bb1f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:17:01.847488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10251a1b077c4066a1898bdab6bb37dd', 'request_id': 'req-1a79ce75-f244-432f-b57b-a0b95a13bb1f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:17:01.922961 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10251a1b077c4066a1898bdab6bb37dd', 'request_id': 'req-1a79ce75-f244-432f-b57b-a0b95a13bb1f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:17:01.944142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] The token cache is being invalidate because project 10251a1b077c4066a1898bdab6bb37dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:17:01.944142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:17:01.944142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-1a79ce75-f244-432f-b57b-a0b95a13bb1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:17:01.944142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1200/2412] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:17:01 2022] DELETE /identity/v3/projects/10251a1b077c4066a1898bdab6bb37dd => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:17:01.945121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f76cfc-9890-45c2-803d-b65912fd3458 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:01.945557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f76cfc-9890-45c2-803d-b65912fd3458 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:01.945805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f76cfc-9890-45c2-803d-b65912fd3458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:01.965314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d8f76cfc-9890-45c2-803d-b65912fd3458 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:01.966805 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d8f76cfc-9890-45c2-803d-b65912fd3458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:01.983914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:17:02.026941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1213/2413] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:17:02.035362 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda765-cb2d-4afc-8519-68bb30d2ffd3 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:02.035654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda765-cb2d-4afc-8519-68bb30d2ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:02.035986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda765-cb2d-4afc-8519-68bb30d2ffd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:02.056945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6eeda765-cb2d-4afc-8519-68bb30d2ffd3 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:02.058213 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6eeda765-cb2d-4afc-8519-68bb30d2ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:02.071336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1201/2414] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:17:02.075614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f76cfc-9890-45c2-803d-b65912fd3458 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:02.086036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f76cfc-9890-45c2-803d-b65912fd3458 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:02.090718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a883820-89b3-406e-9297-14b41274c3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:02.091583 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a883820-89b3-406e-9297-14b41274c3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:02.091868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a883820-89b3-406e-9297-14b41274c3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:02.092146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a883820-89b3-406e-9297-14b41274c3bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:02.092846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a883820-89b3-406e-9297-14b41274c3bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:17:02.093192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a883820-89b3-406e-9297-14b41274c3bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:02.094795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a883820-89b3-406e-9297-14b41274c3bd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:02.107213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1214/2415] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:17:02.115965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeda765-cb2d-4afc-8519-68bb30d2ffd3 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:02.121923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeda765-cb2d-4afc-8519-68bb30d2ffd3 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:02.126652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2aecbc-9fb9-49a8-b3f1-723ef998d6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:02.127266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aecbc-9fb9-49a8-b3f1-723ef998d6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:02.127652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aecbc-9fb9-49a8-b3f1-723ef998d6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:02.127958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aecbc-9fb9-49a8-b3f1-723ef998d6b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:02.128676 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2aecbc-9fb9-49a8-b3f1-723ef998d6b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:02.131325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2aecbc-9fb9-49a8-b3f1-723ef998d6b0 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:02.222147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1202/2416] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:17:02 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:17:02.226772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a883820-89b3-406e-9297-14b41274c3bd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:02.237221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a883820-89b3-406e-9297-14b41274c3bd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:02.296474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ec217-cc79-4574-af98-6edd1dffff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:02.297160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ec217-cc79-4574-af98-6edd1dffff14 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:02.297473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ec217-cc79-4574-af98-6edd1dffff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:02.297758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ec217-cc79-4574-af98-6edd1dffff14 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:02.298430 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ec217-cc79-4574-af98-6edd1dffff14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:02.300353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ec217-cc79-4574-af98-6edd1dffff14 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:02.380912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1215/2417] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:17:02 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:17:02.385286 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2aecbc-9fb9-49a8-b3f1-723ef998d6b0 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:02.396875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2aecbc-9fb9-49a8-b3f1-723ef998d6b0 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:02.401759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46213b8-73b4-40bd-b4a8-b34954f3e10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:02.402388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46213b8-73b4-40bd-b4a8-b34954f3e10c admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:02.402674 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46213b8-73b4-40bd-b4a8-b34954f3e10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:02.402953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46213b8-73b4-40bd-b4a8-b34954f3e10c admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:02.403692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46213b8-73b4-40bd-b4a8-b34954f3e10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:02.405334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46213b8-73b4-40bd-b4a8-b34954f3e10c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:02.417925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1203/2418] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:17:02.423121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ec217-cc79-4574-af98-6edd1dffff14 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:02.434781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ec217-cc79-4574-af98-6edd1dffff14 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:02.439429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c56a5-6053-463d-adc1-7c5a6e36a4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:02.440036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c56a5-6053-463d-adc1-7c5a6e36a4cb admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:02.440317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c56a5-6053-463d-adc1-7c5a6e36a4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:02.440588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c56a5-6053-463d-adc1-7c5a6e36a4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:02.441311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c56a5-6053-463d-adc1-7c5a6e36a4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:02.442950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c56a5-6053-463d-adc1-7c5a6e36a4cb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:02.454975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1216/2419] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:17:02.459130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46213b8-73b4-40bd-b4a8-b34954f3e10c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:02.469451 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46213b8-73b4-40bd-b4a8-b34954f3e10c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:02.474148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68148e2-a2c5-40d0-a8fc-f18186ffb3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:02.475271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68148e2-a2c5-40d0-a8fc-f18186ffb3c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:02.475551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68148e2-a2c5-40d0-a8fc-f18186ffb3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:02.475822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68148e2-a2c5-40d0-a8fc-f18186ffb3c4 admin admin] PATH_INFO: `/v3/projects/bf347b1959b247bcbe003ac3662cb239/users/83ba025a383c433b900a3f4ae64959b3/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:02.492428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68148e2-a2c5-40d0-a8fc-f18186ffb3c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf347b1959b247bcbe003ac3662cb239, user_id=83ba025a383c433b900a3f4ae64959b3, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:02.495815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68148e2-a2c5-40d0-a8fc-f18186ffb3c4 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:02.570076 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1204/2420] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:17:02 2022] PUT /identity/v3/projects/bf347b1959b247bcbe003ac3662cb239/users/83ba025a383c433b900a3f4ae64959b3/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:17:02.575233 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add867f-806d-46b4-9df6-e6a307bcdf62 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:02.575594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add867f-806d-46b4-9df6-e6a307bcdf62 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:02.575937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add867f-806d-46b4-9df6-e6a307bcdf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:02.595473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9add867f-806d-46b4-9df6-e6a307bcdf62 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:02.596731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9add867f-806d-46b4-9df6-e6a307bcdf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:02.656975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1217/2421] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:17:02.667512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68148e2-a2c5-40d0-a8fc-f18186ffb3c4 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:02.669264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f68148e2-a2c5-40d0-a8fc-f18186ffb3c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:02.692058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:17:02.729767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68148e2-a2c5-40d0-a8fc-f18186ffb3c4 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:02.734129 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561d659-b6d6-45cd-88ff-3d770c04aba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:02.734767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561d659-b6d6-45cd-88ff-3d770c04aba8 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:02.735053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561d659-b6d6-45cd-88ff-3d770c04aba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:02.735329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561d659-b6d6-45cd-88ff-3d770c04aba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:02.745364 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561d659-b6d6-45cd-88ff-3d770c04aba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:02.746855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561d659-b6d6-45cd-88ff-3d770c04aba8 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:02.756990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1205/2422] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:17:03.549557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0872a-fdbe-426d-8cc6-8da037b50452 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:03.550145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0872a-fdbe-426d-8cc6-8da037b50452 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:03.550145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0872a-fdbe-426d-8cc6-8da037b50452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:03.571203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-06a0872a-fdbe-426d-8cc6-8da037b50452 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:03.572454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-06a0872a-fdbe-426d-8cc6-8da037b50452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:03.599767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:17:03.640118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1218/2423] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:17:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:17:03.905012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561d659-b6d6-45cd-88ff-3d770c04aba8 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:03.906485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f561d659-b6d6-45cd-88ff-3d770c04aba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:03.982897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561d659-b6d6-45cd-88ff-3d770c04aba8 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:03.992138 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4b5e5-dca9-4967-8846-62afffdf91df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:03.992899 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4b5e5-dca9-4967-8846-62afffdf91df service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:03.993330 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4b5e5-dca9-4967-8846-62afffdf91df service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:03.993605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4b5e5-dca9-4967-8846-62afffdf91df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:04.003078 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4b5e5-dca9-4967-8846-62afffdf91df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:04.004689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4b5e5-dca9-4967-8846-62afffdf91df service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:04.014034 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1206/2424] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:17:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:17:10.404560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139069f6-dd15-40e9-a31c-222375ce7dfc None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:10.405278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139069f6-dd15-40e9-a31c-222375ce7dfc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:10.405593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139069f6-dd15-40e9-a31c-222375ce7dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:10.467135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-139069f6-dd15-40e9-a31c-222375ce7dfc None None] MFA Rules not processed for user `83ba025a383c433b900a3f4ae64959b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:10.468774 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-139069f6-dd15-40e9-a31c-222375ce7dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:10.647829 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1219/2425] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:17:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:17:10.652338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4b5e5-dca9-4967-8846-62afffdf91df service swift] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:10.681646 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4b5e5-dca9-4967-8846-62afffdf91df service swift] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:10.694701 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450776e-0aad-41da-826e-3392a44bc3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:10.695346 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450776e-0aad-41da-826e-3392a44bc3ec admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:10.695635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450776e-0aad-41da-826e-3392a44bc3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:10.695917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450776e-0aad-41da-826e-3392a44bc3ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:10.696605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450776e-0aad-41da-826e-3392a44bc3ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:10.698764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450776e-0aad-41da-826e-3392a44bc3ec admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:10.852543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1207/2426] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:17:10 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:17:10.853442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139069f6-dd15-40e9-a31c-222375ce7dfc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:10.869666 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139069f6-dd15-40e9-a31c-222375ce7dfc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:11.025238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7be31e-7be9-4cf1-9cce-70fc4fb058f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:11.025238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7be31e-7be9-4cf1-9cce-70fc4fb058f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:11.025238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7be31e-7be9-4cf1-9cce-70fc4fb058f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:11.025238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7be31e-7be9-4cf1-9cce-70fc4fb058f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:11.025932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7be31e-7be9-4cf1-9cce-70fc4fb058f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:11.028271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7be31e-7be9-4cf1-9cce-70fc4fb058f8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:11.112507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1220/2427] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:17:10 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:17:11.117444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450776e-0aad-41da-826e-3392a44bc3ec admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:11.137992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450776e-0aad-41da-826e-3392a44bc3ec admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:11.148848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785bf9a-9df8-44a6-9586-b31ff44d3807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:11.150404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785bf9a-9df8-44a6-9586-b31ff44d3807 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:11.150685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785bf9a-9df8-44a6-9586-b31ff44d3807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:11.150967 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785bf9a-9df8-44a6-9586-b31ff44d3807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:11.151688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785bf9a-9df8-44a6-9586-b31ff44d3807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:11.156658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785bf9a-9df8-44a6-9586-b31ff44d3807 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:11.196857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7be31e-7be9-4cf1-9cce-70fc4fb058f8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:11.197723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1208/2428] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:17:11.209874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7be31e-7be9-4cf1-9cce-70fc4fb058f8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:11.227166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc212bfd-e4c7-44de-94f3-5979a8a4b9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:11.227923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc212bfd-e4c7-44de-94f3-5979a8a4b9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:11.228307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc212bfd-e4c7-44de-94f3-5979a8a4b9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:11.228684 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc212bfd-e4c7-44de-94f3-5979a8a4b9b5 admin admin] PATH_INFO: `/v3/projects/40267a64c0a943fea33830788584f799/users/9ea265df98a24fe69d26603db19315b1/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:11.250449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc212bfd-e4c7-44de-94f3-5979a8a4b9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40267a64c0a943fea33830788584f799, user_id=9ea265df98a24fe69d26603db19315b1, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:11.253718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc212bfd-e4c7-44de-94f3-5979a8a4b9b5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:11.335399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1221/2429] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:17:11 2022] PUT /identity/v3/projects/40267a64c0a943fea33830788584f799/users/9ea265df98a24fe69d26603db19315b1/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:17:11.339605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785bf9a-9df8-44a6-9586-b31ff44d3807 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:11.356239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785bf9a-9df8-44a6-9586-b31ff44d3807 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:11.454399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27281c37-2619-46aa-832e-223d8a4f9000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:11.455875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27281c37-2619-46aa-832e-223d8a4f9000 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:11.455875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27281c37-2619-46aa-832e-223d8a4f9000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:11.456476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27281c37-2619-46aa-832e-223d8a4f9000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:11.457557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27281c37-2619-46aa-832e-223d8a4f9000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:11.459423 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27281c37-2619-46aa-832e-223d8a4f9000 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:11.487046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1209/2430] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:17:11.487857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc212bfd-e4c7-44de-94f3-5979a8a4b9b5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:11.505871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc212bfd-e4c7-44de-94f3-5979a8a4b9b5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:11.515557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f520cd0-f98c-4231-922f-03e82e3f251b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:11.516143 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f520cd0-f98c-4231-922f-03e82e3f251b admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:11.516427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f520cd0-f98c-4231-922f-03e82e3f251b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:11.516701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f520cd0-f98c-4231-922f-03e82e3f251b admin admin] PATH_INFO: `/v3/projects/40267a64c0a943fea33830788584f799/users/9ea265df98a24fe69d26603db19315b1/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:11.518832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f520cd0-f98c-4231-922f-03e82e3f251b admin admin] RBAC: Authorizing `identity:create_grant(project_id=40267a64c0a943fea33830788584f799, user_id=9ea265df98a24fe69d26603db19315b1, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:11.523447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f520cd0-f98c-4231-922f-03e82e3f251b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:11.545048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1222/2431] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:17:11 2022] PUT /identity/v3/projects/40267a64c0a943fea33830788584f799/users/9ea265df98a24fe69d26603db19315b1/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:17:18.723678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c00e8-4938-4983-8cdb-2027496445dd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:18.724223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c00e8-4938-4983-8cdb-2027496445dd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:18.724485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c00e8-4938-4983-8cdb-2027496445dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:18.794357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-b71c00e8-4938-4983-8cdb-2027496445dd None None] MFA Rules not processed for user `9ea265df98a24fe69d26603db19315b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:18.794357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b71c00e8-4938-4983-8cdb-2027496445dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:18.797570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:17:18.908763 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1210/2432] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:17:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:17:18.916702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f520cd0-f98c-4231-922f-03e82e3f251b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:18.918677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1f520cd0-f98c-4231-922f-03e82e3f251b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:18.941921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:17:19.012472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f520cd0-f98c-4231-922f-03e82e3f251b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:19.018203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e057e9-5a24-418a-af23-81df746d16cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:19.018915 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e057e9-5a24-418a-af23-81df746d16cd service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:19.019213 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e057e9-5a24-418a-af23-81df746d16cd service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:19.019496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e057e9-5a24-418a-af23-81df746d16cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:19.093567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e057e9-5a24-418a-af23-81df746d16cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:19.095184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e057e9-5a24-418a-af23-81df746d16cd service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:19.126608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1223/2433] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:17:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:17:21.946914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10a3cf-5379-4bd0-a3d0-b52fb50e9c9c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:21.947436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10a3cf-5379-4bd0-a3d0-b52fb50e9c9c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:21.947436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10a3cf-5379-4bd0-a3d0-b52fb50e9c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:21.967670 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8e10a3cf-5379-4bd0-a3d0-b52fb50e9c9c None None] MFA Rules not processed for user `83ba025a383c433b900a3f4ae64959b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:21.973703 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8e10a3cf-5379-4bd0-a3d0-b52fb50e9c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:21.976396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1211/2434] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:17:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:17:23.767073 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cf7ea-fa3f-4651-a1f6-46cca535ccff None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:23.767073 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cf7ea-fa3f-4651-a1f6-46cca535ccff None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:23.767073 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cf7ea-fa3f-4651-a1f6-46cca535ccff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:23.769550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-941cf7ea-fa3f-4651-a1f6-46cca535ccff None None] MFA Rules not processed for user `30f0002d2a5a4950b2f1aa771f33bbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:23.772164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-941cf7ea-fa3f-4651-a1f6-46cca535ccff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:23.835249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1224/2435] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:17:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:17:25.985929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10a3cf-5379-4bd0-a3d0-b52fb50e9c9c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:25.987903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8e10a3cf-5379-4bd0-a3d0-b52fb50e9c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:26.079846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10a3cf-5379-4bd0-a3d0-b52fb50e9c9c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:26.093364 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e7184-02d9-4d1c-91c6-03bfe452f5ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:26.094084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e7184-02d9-4d1c-91c6-03bfe452f5ee service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:26.094365 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e7184-02d9-4d1c-91c6-03bfe452f5ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:26.095033 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e7184-02d9-4d1c-91c6-03bfe452f5ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:26.097209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ca2e7184-02d9-4d1c-91c6-03bfe452f5ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:26.125715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e7184-02d9-4d1c-91c6-03bfe452f5ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:26.127268 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e7184-02d9-4d1c-91c6-03bfe452f5ee service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:26.141874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1212/2436] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:17:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:17:27.942710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8aff1-983c-4824-a7a1-6c7eca1ec5a8 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:27.943205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8aff1-983c-4824-a7a1-6c7eca1ec5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:27.943325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8aff1-983c-4824-a7a1-6c7eca1ec5a8 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:27.944760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1225/2437] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:17:27.951299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228c0d1-e3f1-4009-985f-eba67d6db102 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:27.951778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228c0d1-e3f1-4009-985f-eba67d6db102 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:27.952147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228c0d1-e3f1-4009-985f-eba67d6db102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:27.978515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a228c0d1-e3f1-4009-985f-eba67d6db102 None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:27.979794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a228c0d1-e3f1-4009-985f-eba67d6db102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:27.989630 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1213/2438] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:17:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:17:28.001846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8aff1-983c-4824-a7a1-6c7eca1ec5a8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:28.003551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d2e8aff1-983c-4824-a7a1-6c7eca1ec5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:28.065698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8aff1-983c-4824-a7a1-6c7eca1ec5a8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:28.070010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db567955-19a4-4105-b71e-ae54928ac17a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47f1552e73648ceb8a421c0543c2974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:28.070654 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567955-19a4-4105-b71e-ae54928ac17a service glance] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:28.070942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567955-19a4-4105-b71e-ae54928ac17a service glance] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:28.071223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567955-19a4-4105-b71e-ae54928ac17a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:28.082373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db567955-19a4-4105-b71e-ae54928ac17a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:28.084311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db567955-19a4-4105-b71e-ae54928ac17a service glance] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:28.090298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:17:28.096045 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1226/2439] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:17:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:17:28.261459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a54221-7bd5-42d1-8015-ab34f799ef77 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:28.261772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a54221-7bd5-42d1-8015-ab34f799ef77 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:28.262053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a54221-7bd5-42d1-8015-ab34f799ef77 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:28.263448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1214/2440] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:17:28.268492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef3fa3-6fa4-4f1b-bd61-3e855521355d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:28.268861 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef3fa3-6fa4-4f1b-bd61-3e855521355d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:28.269239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef3fa3-6fa4-4f1b-bd61-3e855521355d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:28.291911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3eef3fa3-6fa4-4f1b-bd61-3e855521355d None None] MFA Rules not processed for user `b038336582a0435dbd47ad2c09cd23e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:28.293173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3eef3fa3-6fa4-4f1b-bd61-3e855521355d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:28.299822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1227/2441] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:17:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:17:28.311594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a54221-7bd5-42d1-8015-ab34f799ef77 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:28.324162 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a54221-7bd5-42d1-8015-ab34f799ef77 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:28.327892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a824a5-bd1c-4655-954f-72fafafa3670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:28.328513 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a824a5-bd1c-4655-954f-72fafafa3670 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:28.328797 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a824a5-bd1c-4655-954f-72fafafa3670 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:28.329110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a824a5-bd1c-4655-954f-72fafafa3670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:28.337273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a824a5-bd1c-4655-954f-72fafafa3670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:28.338707 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a824a5-bd1c-4655-954f-72fafafa3670 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:28.347005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1215/2442] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:17:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:17:28.435069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70f225-3833-4276-8d75-c727ef4c3798 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:28.435355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70f225-3833-4276-8d75-c727ef4c3798 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:28.435641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70f225-3833-4276-8d75-c727ef4c3798 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:28.437109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1228/2443] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:17:28.442149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aff4ab-cfc8-4b7f-9005-6705dc2a7840 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:28.442437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aff4ab-cfc8-4b7f-9005-6705dc2a7840 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:28.442711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aff4ab-cfc8-4b7f-9005-6705dc2a7840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:28.471753 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-39aff4ab-cfc8-4b7f-9005-6705dc2a7840 None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:28.473893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-39aff4ab-cfc8-4b7f-9005-6705dc2a7840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:28.489666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1216/2444] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:17:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:17:28.501636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70f225-3833-4276-8d75-c727ef4c3798 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:28.514018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70f225-3833-4276-8d75-c727ef4c3798 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:28.517716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fce0b6-2369-4f11-a065-4dec2bce5728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:28.518327 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fce0b6-2369-4f11-a065-4dec2bce5728 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:28.518613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fce0b6-2369-4f11-a065-4dec2bce5728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:28.518889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fce0b6-2369-4f11-a065-4dec2bce5728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:28.532906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fce0b6-2369-4f11-a065-4dec2bce5728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:28.534406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fce0b6-2369-4f11-a065-4dec2bce5728 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:28.546000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1229/2445] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:17:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:17:28.587948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aff4ab-cfc8-4b7f-9005-6705dc2a7840 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:28.597621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aff4ab-cfc8-4b7f-9005-6705dc2a7840 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:28.604212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b6f99-3b2d-4e90-a373-d3a7fa360d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:28.604857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b6f99-3b2d-4e90-a373-d3a7fa360d01 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:28.605159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b6f99-3b2d-4e90-a373-d3a7fa360d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:28.605433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b6f99-3b2d-4e90-a373-d3a7fa360d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:28.607209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7c0b6f99-3b2d-4e90-a373-d3a7fa360d01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:28.621813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b6f99-3b2d-4e90-a373-d3a7fa360d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:28.623248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b6f99-3b2d-4e90-a373-d3a7fa360d01 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:28.634810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1217/2446] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:17:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:17:38.540847 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=9 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:17:38.629351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=74 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:17:41.820277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fce0b6-2369-4f11-a065-4dec2bce5728 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:41.821841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-15fce0b6-2369-4f11-a065-4dec2bce5728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:41.878501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fce0b6-2369-4f11-a065-4dec2bce5728 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:41.882121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23fe16-9fb4-42dc-a928-018045713aa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:41.882736 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23fe16-9fb4-42dc-a928-018045713aa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:41.883018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23fe16-9fb4-42dc-a928-018045713aa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:41.883293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23fe16-9fb4-42dc-a928-018045713aa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:41.885061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cb23fe16-9fb4-42dc-a928-018045713aa5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:41.942385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23fe16-9fb4-42dc-a928-018045713aa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:41.943815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23fe16-9fb4-42dc-a928-018045713aa5 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:41.953239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1230/2447] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:17:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:17:43.882729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0379c-ec12-4560-949a-97c337167a55 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:43.883215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0379c-ec12-4560-949a-97c337167a55 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:43.883346 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0379c-ec12-4560-949a-97c337167a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:43.903192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-79e0379c-ec12-4560-949a-97c337167a55 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:17:43.904466 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-79e0379c-ec12-4560-949a-97c337167a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:17:43.979026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1218/2448] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:17:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:17:43.991536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23fe16-9fb4-42dc-a928-018045713aa5 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:17:44.059239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23fe16-9fb4-42dc-a928-018045713aa5 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:17:44.064575 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f33ef-6d32-4ec0-92e1-eea2ea093c5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:17:44.065247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f33ef-6d32-4ec0-92e1-eea2ea093c5e service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:17:44.065513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f33ef-6d32-4ec0-92e1-eea2ea093c5e service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:17:44.065788 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f33ef-6d32-4ec0-92e1-eea2ea093c5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:17:44.075150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f33ef-6d32-4ec0-92e1-eea2ea093c5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:17:44.076735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f33ef-6d32-4ec0-92e1-eea2ea093c5e service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:17:44.085148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1231/2449] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:17:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 18:17:53.970979 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=25 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:17:54.082153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=80 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:18:02.465431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0379c-ec12-4560-949a-97c337167a55 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:02.475788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0379c-ec12-4560-949a-97c337167a55 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:02.479541 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab95d07-06b3-4aa6-a89f-7519e825d7be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:02.480266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab95d07-06b3-4aa6-a89f-7519e825d7be service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:02.480550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab95d07-06b3-4aa6-a89f-7519e825d7be service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:02.480824 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab95d07-06b3-4aa6-a89f-7519e825d7be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:02.482749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fab95d07-06b3-4aa6-a89f-7519e825d7be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:02.493052 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab95d07-06b3-4aa6-a89f-7519e825d7be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:02.495294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab95d07-06b3-4aa6-a89f-7519e825d7be service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:02.504429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1219/2450] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:03.203160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f33ef-6d32-4ec0-92e1-eea2ea093c5e service swift] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:03.204639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-de9f33ef-6d32-4ec0-92e1-eea2ea093c5e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:03.277519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f33ef-6d32-4ec0-92e1-eea2ea093c5e service swift] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:03.278996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab95d07-06b3-4aa6-a89f-7519e825d7be service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:03.280523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fab95d07-06b3-4aa6-a89f-7519e825d7be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:03.282627 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bf24c-50a9-42fb-b7e9-51155f2c15b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:03.283255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bf24c-50a9-42fb-b7e9-51155f2c15b7 service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:03.283540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bf24c-50a9-42fb-b7e9-51155f2c15b7 service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:03.283820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bf24c-50a9-42fb-b7e9-51155f2c15b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:03.285555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f85bf24c-50a9-42fb-b7e9-51155f2c15b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:03.297892 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bf24c-50a9-42fb-b7e9-51155f2c15b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:03.299563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bf24c-50a9-42fb-b7e9-51155f2c15b7 service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:03.310056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1232/2451] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:03.348445 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab95d07-06b3-4aa6-a89f-7519e825d7be service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:03.353153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:03.353794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:03.354088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:03.354368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] PATH_INFO: `/v3/users/bd62d8053b544f8bb1ca173cd007dbef` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:03.355564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd62d8053b544f8bb1ca173cd007dbef)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:03.357555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:03.547686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd62d8053b544f8bb1ca173cd007dbef'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:03.555151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd62d8053b544f8bb1ca173cd007dbef'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:03.641000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd62d8053b544f8bb1ca173cd007dbef'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:03.641566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd62d8053b544f8bb1ca173cd007dbef'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:03.654013 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd62d8053b544f8bb1ca173cd007dbef'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:03.659873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1220/2452] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:18:03 2022] DELETE /identity/v3/users/bd62d8053b544f8bb1ca173cd007dbef => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:18:03.665298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35c7e8-79d9-4102-8415-b3a1cc0030bf None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:03.665585 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35c7e8-79d9-4102-8415-b3a1cc0030bf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:03.665862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35c7e8-79d9-4102-8415-b3a1cc0030bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:03.686119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6e35c7e8-79d9-4102-8415-b3a1cc0030bf None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:18:03.687390 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6e35c7e8-79d9-4102-8415-b3a1cc0030bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:03.749132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1233/2453] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:18:03.758782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:03.760204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:03.819988 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35e341-94aa-471e-a3b4-3eb6fc56638d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:03.824092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a92c8e-90b6-49bb-aaa7-f2dd0c647709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:03.824317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92c8e-90b6-49bb-aaa7-f2dd0c647709 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:03.824598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92c8e-90b6-49bb-aaa7-f2dd0c647709 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:03.824885 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92c8e-90b6-49bb-aaa7-f2dd0c647709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:03.834242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a92c8e-90b6-49bb-aaa7-f2dd0c647709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:03.835748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a92c8e-90b6-49bb-aaa7-f2dd0c647709 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:03.845661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1221/2454] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:04.273655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35c7e8-79d9-4102-8415-b3a1cc0030bf None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:04.275446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6e35c7e8-79d9-4102-8415-b3a1cc0030bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:04.296513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35c7e8-79d9-4102-8415-b3a1cc0030bf None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:04.305557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:04.306704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:04.306704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:04.309630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] PATH_INFO: `/v3/projects/a55e72ee6d5f4525ad37b9fb366c7506` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:04.309630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a55e72ee6d5f4525ad37b9fb366c7506)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:04.313778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:04.425831 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a55e72ee6d5f4525ad37b9fb366c7506', 'request_id': 'req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:04.431668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a55e72ee6d5f4525ad37b9fb366c7506', 'request_id': 'req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:04.438136 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a55e72ee6d5f4525ad37b9fb366c7506', 'request_id': 'req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:04.539268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] The token cache is being invalidate because project a55e72ee6d5f4525ad37b9fb366c7506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:18:04.539566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:04.540223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6d0d55ac-d64c-4101-ab1b-1ad1a4ac714c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:04.541844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1234/2455] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:18:04 2022] DELETE /identity/v3/projects/a55e72ee6d5f4525ad37b9fb366c7506 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:18:04.546688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10512087-3814-435c-b560-13658dbdc5d2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:04.546986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10512087-3814-435c-b560-13658dbdc5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:04.547269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10512087-3814-435c-b560-13658dbdc5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:04.567118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-10512087-3814-435c-b560-13658dbdc5d2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:18:04.569918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-10512087-3814-435c-b560-13658dbdc5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:04.573597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:18:04.629244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1222/2456] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:18:04.637357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7030e33-e07a-4f9d-89c1-882ed6402620 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:04.637646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7030e33-e07a-4f9d-89c1-882ed6402620 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:04.637931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7030e33-e07a-4f9d-89c1-882ed6402620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:04.658301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f7030e33-e07a-4f9d-89c1-882ed6402620 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:18:04.661677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f7030e33-e07a-4f9d-89c1-882ed6402620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:04.667726 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1235/2457] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:18:04.671977 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10512087-3814-435c-b560-13658dbdc5d2 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:04.682099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10512087-3814-435c-b560-13658dbdc5d2 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:04.687325 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea483ef-76ee-4382-8e13-3476b3152b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:04.687937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea483ef-76ee-4382-8e13-3476b3152b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:04.688336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea483ef-76ee-4382-8e13-3476b3152b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:04.688558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea483ef-76ee-4382-8e13-3476b3152b47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:04.689304 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea483ef-76ee-4382-8e13-3476b3152b47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:18:04.689609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea483ef-76ee-4382-8e13-3476b3152b47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:04.691200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea483ef-76ee-4382-8e13-3476b3152b47 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:04.704372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1223/2458] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:18:04.708711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7030e33-e07a-4f9d-89c1-882ed6402620 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:04.719664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7030e33-e07a-4f9d-89c1-882ed6402620 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:04.724896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb7922-ec6a-43a6-bd64-ae160087631c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:04.725574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb7922-ec6a-43a6-bd64-ae160087631c admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:04.725860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb7922-ec6a-43a6-bd64-ae160087631c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:04.726142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb7922-ec6a-43a6-bd64-ae160087631c admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:04.726879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb7922-ec6a-43a6-bd64-ae160087631c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:04.729010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb7922-ec6a-43a6-bd64-ae160087631c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:04.734631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:18:04.866955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1236/2459] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:18:04 2022] POST /identity/v3/projects => generated 391 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:18:04.870989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea483ef-76ee-4382-8e13-3476b3152b47 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:04.880928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea483ef-76ee-4382-8e13-3476b3152b47 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:04.938151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a06333-a19b-4792-95f4-63f963b2d73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:04.938976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a06333-a19b-4792-95f4-63f963b2d73d admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:04.939260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a06333-a19b-4792-95f4-63f963b2d73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:04.939534 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a06333-a19b-4792-95f4-63f963b2d73d admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:04.940202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a06333-a19b-4792-95f4-63f963b2d73d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:04.942255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a06333-a19b-4792-95f4-63f963b2d73d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:05.026126 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1224/2460] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:18:04 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:18:05.030367 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb7922-ec6a-43a6-bd64-ae160087631c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:05.041429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb7922-ec6a-43a6-bd64-ae160087631c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:05.046195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb039c-8a1a-4736-82f5-143f8b294e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:05.048524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb039c-8a1a-4736-82f5-143f8b294e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:05.048524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb039c-8a1a-4736-82f5-143f8b294e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:05.048524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb039c-8a1a-4736-82f5-143f8b294e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:05.048524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb039c-8a1a-4736-82f5-143f8b294e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:05.049664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb039c-8a1a-4736-82f5-143f8b294e08 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:05.059854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1237/2461] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:18:05.063650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a06333-a19b-4792-95f4-63f963b2d73d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:05.074422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a06333-a19b-4792-95f4-63f963b2d73d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:05.079274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b63f5b-34d4-4a3f-8128-189b9169e707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:05.079892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b63f5b-34d4-4a3f-8128-189b9169e707 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:05.080181 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b63f5b-34d4-4a3f-8128-189b9169e707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:05.080460 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b63f5b-34d4-4a3f-8128-189b9169e707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:05.081205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b63f5b-34d4-4a3f-8128-189b9169e707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:05.082814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b63f5b-34d4-4a3f-8128-189b9169e707 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:05.094232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1225/2462] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:18:05.098438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb039c-8a1a-4736-82f5-143f8b294e08 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:05.109569 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb039c-8a1a-4736-82f5-143f8b294e08 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:05.114313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c35a5d-e3a0-4325-a3af-cb004fb6cdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:05.114979 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c35a5d-e3a0-4325-a3af-cb004fb6cdae admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:05.115259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c35a5d-e3a0-4325-a3af-cb004fb6cdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:05.115531 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c35a5d-e3a0-4325-a3af-cb004fb6cdae admin admin] PATH_INFO: `/v3/projects/6cce8bfa6dc241739e1fe5e072a4e51a/users/70edc2ff78c846f5b6adf4e3bcdec70b/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:05.130306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c35a5d-e3a0-4325-a3af-cb004fb6cdae admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cce8bfa6dc241739e1fe5e072a4e51a, user_id=70edc2ff78c846f5b6adf4e3bcdec70b, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:05.133485 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c35a5d-e3a0-4325-a3af-cb004fb6cdae admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:05.206446 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1238/2463] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:18:05 2022] PUT /identity/v3/projects/6cce8bfa6dc241739e1fe5e072a4e51a/users/70edc2ff78c846f5b6adf4e3bcdec70b/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:18:05.218896 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e20f6-233d-4a36-8b36-098137921746 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:05.219147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e20f6-233d-4a36-8b36-098137921746 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:05.219430 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e20f6-233d-4a36-8b36-098137921746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:05.248724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8b6e20f6-233d-4a36-8b36-098137921746 None None] MFA Rules not processed for user `70edc2ff78c846f5b6adf4e3bcdec70b`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:18:05.250609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8b6e20f6-233d-4a36-8b36-098137921746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:05.317506 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1226/2464] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:18:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:18:05.328557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c35a5d-e3a0-4325-a3af-cb004fb6cdae admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:05.330340 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-81c35a5d-e3a0-4325-a3af-cb004fb6cdae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:05.393875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c35a5d-e3a0-4325-a3af-cb004fb6cdae admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:05.398424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9c936-f060-4682-9474-a5a7b861e6ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:05.399252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c936-f060-4682-9474-a5a7b861e6ba service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:05.399438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c936-f060-4682-9474-a5a7b861e6ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:05.399712 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c936-f060-4682-9474-a5a7b861e6ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:05.409513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9c936-f060-4682-9474-a5a7b861e6ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:05.411103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9c936-f060-4682-9474-a5a7b861e6ba service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:05.422898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1239/2465] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:06.632842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e20f6-233d-4a36-8b36-098137921746 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:06.634406 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8b6e20f6-233d-4a36-8b36-098137921746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:06.640707 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:18:06.703340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e20f6-233d-4a36-8b36-098137921746 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:06.707227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2218f18-44b9-4f65-b35c-054311025fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:06.707846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2218f18-44b9-4f65-b35c-054311025fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:06.708370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2218f18-44b9-4f65-b35c-054311025fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:06.708644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2218f18-44b9-4f65-b35c-054311025fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:06.713270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b2218f18-44b9-4f65-b35c-054311025fd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:06.724328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2218f18-44b9-4f65-b35c-054311025fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:06.726424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2218f18-44b9-4f65-b35c-054311025fd2 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:06.735302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1227/2466] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:07.617733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9c936-f060-4682-9474-a5a7b861e6ba service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:07.621048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-73a9c936-f060-4682-9474-a5a7b861e6ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:07.705196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9c936-f060-4682-9474-a5a7b861e6ba service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:07.711855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42342d-d150-4810-9191-7d9aa5658ae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:07.712499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42342d-d150-4810-9191-7d9aa5658ae3 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:07.712785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42342d-d150-4810-9191-7d9aa5658ae3 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:07.713107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42342d-d150-4810-9191-7d9aa5658ae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:07.716171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ce42342d-d150-4810-9191-7d9aa5658ae3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:07.730134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42342d-d150-4810-9191-7d9aa5658ae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:07.731907 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42342d-d150-4810-9191-7d9aa5658ae3 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:07.746193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1240/2467] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:16.869701 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=24 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:18:17.739841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=81,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:18:22.722415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2218f18-44b9-4f65-b35c-054311025fd2 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:22.735675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2218f18-44b9-4f65-b35c-054311025fd2 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:22.742158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a73d28-0582-40bd-9ae4-59e4b02a7482 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:22.742778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73d28-0582-40bd-9ae4-59e4b02a7482 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:22.743065 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73d28-0582-40bd-9ae4-59e4b02a7482 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:22.743441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73d28-0582-40bd-9ae4-59e4b02a7482 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:22.745295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f9a73d28-0582-40bd-9ae4-59e4b02a7482 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:22.758376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a73d28-0582-40bd-9ae4-59e4b02a7482 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:22.760061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a73d28-0582-40bd-9ae4-59e4b02a7482 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:22.771110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1228/2468] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:30.349721 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42342d-d150-4810-9191-7d9aa5658ae3 service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:30.351394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ce42342d-d150-4810-9191-7d9aa5658ae3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:30.362996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42342d-d150-4810-9191-7d9aa5658ae3 service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:30.367329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5b984-138f-4113-8188-aa9a68c165fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:30.368042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5b984-138f-4113-8188-aa9a68c165fb service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:30.368325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5b984-138f-4113-8188-aa9a68c165fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:30.368602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5b984-138f-4113-8188-aa9a68c165fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:30.370479 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5da5b984-138f-4113-8188-aa9a68c165fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:30.438853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5b984-138f-4113-8188-aa9a68c165fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:30.440324 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5b984-138f-4113-8188-aa9a68c165fb service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:30.451022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1241/2469] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:31.381178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8ab58-9ac1-4a9e-a439-d1c45a2d5e9c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:31.382226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8ab58-9ac1-4a9e-a439-d1c45a2d5e9c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:31.382924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8ab58-9ac1-4a9e-a439-d1c45a2d5e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:31.404474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-dad8ab58-9ac1-4a9e-a439-d1c45a2d5e9c None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:18:31.405752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-dad8ab58-9ac1-4a9e-a439-d1c45a2d5e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:31.470334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1229/2470] 10.222.0.86 () {58 vars in 967 bytes} [Thu Jul 28 18:18:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:18:31.498803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5b984-138f-4113-8188-aa9a68c165fb service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:31.500530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5da5b984-138f-4113-8188-aa9a68c165fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:31.572800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5b984-138f-4113-8188-aa9a68c165fb service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:31.577923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e76c8-09d8-4885-98c7-a5fbd2b77dc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:31.578545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e76c8-09d8-4885-98c7-a5fbd2b77dc6 service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:31.578829 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e76c8-09d8-4885-98c7-a5fbd2b77dc6 service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:31.579106 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e76c8-09d8-4885-98c7-a5fbd2b77dc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:31.588476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e76c8-09d8-4885-98c7-a5fbd2b77dc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:31.590073 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e76c8-09d8-4885-98c7-a5fbd2b77dc6 service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:31.600242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1242/2471] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:18:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:18:31.613712 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8ab58-9ac1-4a9e-a439-d1c45a2d5e9c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:31.702288 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8ab58-9ac1-4a9e-a439-d1c45a2d5e9c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:31.714478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:31.714478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:31.714478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:31.714478 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] PATH_INFO: `/v3/users/70edc2ff78c846f5b6adf4e3bcdec70b` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:31.714924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70edc2ff78c846f5b6adf4e3bcdec70b)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:31.714924 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:31.958583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70edc2ff78c846f5b6adf4e3bcdec70b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:31.991278 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70edc2ff78c846f5b6adf4e3bcdec70b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:32.074961 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70edc2ff78c846f5b6adf4e3bcdec70b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:32.075622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70edc2ff78c846f5b6adf4e3bcdec70b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:32.089258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70edc2ff78c846f5b6adf4e3bcdec70b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:32.095438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1230/2472] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:18:31 2022] DELETE /identity/v3/users/70edc2ff78c846f5b6adf4e3bcdec70b => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:18:32.102108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6534f1-8467-40d0-9450-25804413c687 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:32.102108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6534f1-8467-40d0-9450-25804413c687 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:32.102108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6534f1-8467-40d0-9450-25804413c687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:32.124825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0f6534f1-8467-40d0-9450-25804413c687 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:18:32.126106 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0f6534f1-8467-40d0-9450-25804413c687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:32.187703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1243/2473] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:18:32.198067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:32.199690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:32.216629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:18:32.262872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84735253-a2a2-4a83-a64c-6aa674d8e914 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:32.266991 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855b759-c6b8-4bb6-b0f4-b2985e28a009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:32.275919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855b759-c6b8-4bb6-b0f4-b2985e28a009 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:32.275919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855b759-c6b8-4bb6-b0f4-b2985e28a009 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:32.275919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855b759-c6b8-4bb6-b0f4-b2985e28a009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:32.278322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855b759-c6b8-4bb6-b0f4-b2985e28a009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:32.279755 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855b759-c6b8-4bb6-b0f4-b2985e28a009 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:32.289878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1231/2474] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:32.724933 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6534f1-8467-40d0-9450-25804413c687 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:32.730837 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0f6534f1-8467-40d0-9450-25804413c687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:32.764259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:18:32.823870 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6534f1-8467-40d0-9450-25804413c687 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:32.830689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e92c83-4af6-49bb-98bf-85ee80ac46f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:32.831738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e92c83-4af6-49bb-98bf-85ee80ac46f8 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:32.832181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e92c83-4af6-49bb-98bf-85ee80ac46f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:32.832753 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e92c83-4af6-49bb-98bf-85ee80ac46f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:32.835530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-89e92c83-4af6-49bb-98bf-85ee80ac46f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:32.856719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855b759-c6b8-4bb6-b0f4-b2985e28a009 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:32.856719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d855b759-c6b8-4bb6-b0f4-b2985e28a009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:32.869244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e92c83-4af6-49bb-98bf-85ee80ac46f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:32.869244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e92c83-4af6-49bb-98bf-85ee80ac46f8 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:32.875501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855b759-c6b8-4bb6-b0f4-b2985e28a009 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:32.884302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1244/2475] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:32.892428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:32.893625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:32.893625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:32.893826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] PATH_INFO: `/v3/projects/6cce8bfa6dc241739e1fe5e072a4e51a` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:32.895646 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cce8bfa6dc241739e1fe5e072a4e51a)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:32.901494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:33.065127 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cce8bfa6dc241739e1fe5e072a4e51a', 'request_id': 'req-34a809c9-2877-45ed-80b3-c4450ec1b046'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:33.069590 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cce8bfa6dc241739e1fe5e072a4e51a', 'request_id': 'req-34a809c9-2877-45ed-80b3-c4450ec1b046'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:33.135585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cce8bfa6dc241739e1fe5e072a4e51a', 'request_id': 'req-34a809c9-2877-45ed-80b3-c4450ec1b046'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:33.149048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] The token cache is being invalidate because project 6cce8bfa6dc241739e1fe5e072a4e51a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:18:33.149048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:33.150239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:18:33.152114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1232/2476] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:18:32 2022] DELETE /identity/v3/projects/6cce8bfa6dc241739e1fe5e072a4e51a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:18:36.554564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e92c83-4af6-49bb-98bf-85ee80ac46f8 service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:36.556469 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-89e92c83-4af6-49bb-98bf-85ee80ac46f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:36.567655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e92c83-4af6-49bb-98bf-85ee80ac46f8 service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:36.615661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34aec4-a1b6-4080-9bde-a81082c0a167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:36.616274 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34aec4-a1b6-4080-9bde-a81082c0a167 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:36.616553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34aec4-a1b6-4080-9bde-a81082c0a167 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:36.616827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34aec4-a1b6-4080-9bde-a81082c0a167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:36.618660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ea34aec4-a1b6-4080-9bde-a81082c0a167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:36.679181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34aec4-a1b6-4080-9bde-a81082c0a167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:36.681232 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34aec4-a1b6-4080-9bde-a81082c0a167 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:36.690632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1245/2477] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:39.939992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:18:39.941444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:40.014923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a809c9-2877-45ed-80b3-c4450ec1b046 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:18:40.020469 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f8208-8835-41b3-92d6-d13a5387a3fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:18:40.021117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f8208-8835-41b3-92d6-d13a5387a3fc service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:18:40.021403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f8208-8835-41b3-92d6-d13a5387a3fc service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:18:40.021679 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f8208-8835-41b3-92d6-d13a5387a3fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:18:40.023396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-899f8208-8835-41b3-92d6-d13a5387a3fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:18:40.057189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:18:40.081661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f8208-8835-41b3-92d6-d13a5387a3fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:18:40.083241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f8208-8835-41b3-92d6-d13a5387a3fc service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:18:40.092986 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1233/2478] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:18:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:18:46.686544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=73 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:18:50.218517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=12 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:19:01.457329 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34aec4-a1b6-4080-9bde-a81082c0a167 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:19:01.458891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ea34aec4-a1b6-4080-9bde-a81082c0a167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:19:01.475558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34aec4-a1b6-4080-9bde-a81082c0a167 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:19:01.481014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535ee22-a07b-4676-8a47-65057f0f1aaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:19:01.481632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535ee22-a07b-4676-8a47-65057f0f1aaa service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:19:01.481963 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535ee22-a07b-4676-8a47-65057f0f1aaa service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:19:01.482246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535ee22-a07b-4676-8a47-65057f0f1aaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:19:01.484098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b535ee22-a07b-4676-8a47-65057f0f1aaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:19:01.543674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535ee22-a07b-4676-8a47-65057f0f1aaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:19:01.545313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535ee22-a07b-4676-8a47-65057f0f1aaa service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:19:01.555157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1246/2479] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:19:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:19:10.935946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1080d4c-c9ce-404e-86fd-ea6855aefccf None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:19:10.936484 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1080d4c-c9ce-404e-86fd-ea6855aefccf None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:19:10.936484 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1080d4c-c9ce-404e-86fd-ea6855aefccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:19:10.955061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a1080d4c-c9ce-404e-86fd-ea6855aefccf None None] MFA Rules not processed for user `5ef698c4f57f44d387550921a288d7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:19:10.956270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a1080d4c-c9ce-404e-86fd-ea6855aefccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:19:10.963749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1234/2480] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:19:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:19:11.550361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=28 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:19:20.949860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:19:55.533545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700d3bf-01ac-49fa-9aad-ef66e18437ae None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:19:55.534067 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700d3bf-01ac-49fa-9aad-ef66e18437ae None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:19:55.534176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700d3bf-01ac-49fa-9aad-ef66e18437ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:19:55.553862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6700d3bf-01ac-49fa-9aad-ef66e18437ae None None] MFA Rules not processed for user `ca7d773290a9483d91ebefad341addee`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:19:55.555079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6700d3bf-01ac-49fa-9aad-ef66e18437ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:19:55.610519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1247/2481] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:19:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:19:55.916317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1080d4c-c9ce-404e-86fd-ea6855aefccf None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:19:55.917759 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a1080d4c-c9ce-404e-86fd-ea6855aefccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:19:55.929649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1080d4c-c9ce-404e-86fd-ea6855aefccf None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:19:55.933252 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a6a6a-9ccf-49d0-8276-02d2baa7e59e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:19:55.933867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a6a6a-9ccf-49d0-8276-02d2baa7e59e service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:19:55.934152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a6a6a-9ccf-49d0-8276-02d2baa7e59e service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:19:55.934426 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a6a6a-9ccf-49d0-8276-02d2baa7e59e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:19:55.936366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cf8a6a6a-9ccf-49d0-8276-02d2baa7e59e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:19:55.946466 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a6a6a-9ccf-49d0-8276-02d2baa7e59e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:19:55.947907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a6a6a-9ccf-49d0-8276-02d2baa7e59e service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:19:55.956944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1235/2482] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:19:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:20:05.603192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=25 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:20:05.953110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=6 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:20:31.437669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7904df-fb4f-4bff-90e6-2317f8374200 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:31.438193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7904df-fb4f-4bff-90e6-2317f8374200 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:31.438193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7904df-fb4f-4bff-90e6-2317f8374200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:31.466642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-cb7904df-fb4f-4bff-90e6-2317f8374200 None None] MFA Rules not processed for user `b9874df18af245688db1137b6f1b0bff`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:20:31.467853 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cb7904df-fb4f-4bff-90e6-2317f8374200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:31.491315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1248/2483] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:20:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:20:36.538495 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a6a6a-9ccf-49d0-8276-02d2baa7e59e service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:36.539931 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cf8a6a6a-9ccf-49d0-8276-02d2baa7e59e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:36.635421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7904df-fb4f-4bff-90e6-2317f8374200 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:36.636940 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cb7904df-fb4f-4bff-90e6-2317f8374200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:36.678584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a6a6a-9ccf-49d0-8276-02d2baa7e59e service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:36.684304 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d442e67-a778-460d-b355-e1cd15309bd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:36.684912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d442e67-a778-460d-b355-e1cd15309bd8 service placement] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:36.685235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d442e67-a778-460d-b355-e1cd15309bd8 service placement] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:36.685844 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d442e67-a778-460d-b355-e1cd15309bd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:36.687428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2d442e67-a778-460d-b355-e1cd15309bd8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:36.699419 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d442e67-a778-460d-b355-e1cd15309bd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:36.701056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d442e67-a778-460d-b355-e1cd15309bd8 service placement] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:36.708267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7904df-fb4f-4bff-90e6-2317f8374200 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:36.712019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1236/2484] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:20:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:20:36.714757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd2d00-76c3-41fe-ad8f-37ee6c402842 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:36.715428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd2d00-76c3-41fe-ad8f-37ee6c402842 service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:36.715719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd2d00-76c3-41fe-ad8f-37ee6c402842 service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:36.716532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd2d00-76c3-41fe-ad8f-37ee6c402842 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:36.725572 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd2d00-76c3-41fe-ad8f-37ee6c402842 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:36.727157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd2d00-76c3-41fe-ad8f-37ee6c402842 service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:36.744437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1249/2485] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:20:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:20:38.099067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d442e67-a778-460d-b355-e1cd15309bd8 service placement] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:38.109370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d442e67-a778-460d-b355-e1cd15309bd8 service placement] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:38.115048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cece77f-3009-48da-b986-9e9273c97356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:38.115048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece77f-3009-48da-b986-9e9273c97356 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:38.115048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece77f-3009-48da-b986-9e9273c97356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:38.115048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece77f-3009-48da-b986-9e9273c97356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:38.116261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2cece77f-3009-48da-b986-9e9273c97356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:38.205113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cece77f-3009-48da-b986-9e9273c97356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:38.206609 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cece77f-3009-48da-b986-9e9273c97356 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:38.225147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:20:38.228840 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1237/2486] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:20:42.350196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd2d00-76c3-41fe-ad8f-37ee6c402842 service placement] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:42.351777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3fcd2d00-76c3-41fe-ad8f-37ee6c402842 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:42.367019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd2d00-76c3-41fe-ad8f-37ee6c402842 service placement] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:42.371734 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6abfd-ec9b-4e25-94f1-d047cfb90a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:42.373148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6abfd-ec9b-4e25-94f1-d047cfb90a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:42.373148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6abfd-ec9b-4e25-94f1-d047cfb90a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:42.373148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6abfd-ec9b-4e25-94f1-d047cfb90a29 admin admin] PATH_INFO: `/v3/users/5ef698c4f57f44d387550921a288d7ab` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:42.374150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6abfd-ec9b-4e25-94f1-d047cfb90a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ef698c4f57f44d387550921a288d7ab)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:42.376061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6abfd-ec9b-4e25-94f1-d047cfb90a29 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:42.561171 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c1f6abfd-ec9b-4e25-94f1-d047cfb90a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ef698c4f57f44d387550921a288d7ab'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:20:42.561743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c1f6abfd-ec9b-4e25-94f1-d047cfb90a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ef698c4f57f44d387550921a288d7ab'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:20:42.568833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c1f6abfd-ec9b-4e25-94f1-d047cfb90a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ef698c4f57f44d387550921a288d7ab'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:20:42.639458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c1f6abfd-ec9b-4e25-94f1-d047cfb90a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ef698c4f57f44d387550921a288d7ab'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:20:42.640092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c1f6abfd-ec9b-4e25-94f1-d047cfb90a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ef698c4f57f44d387550921a288d7ab'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:20:42.657766 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1250/2487] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:20:42 2022] DELETE /identity/v3/users/5ef698c4f57f44d387550921a288d7ab => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:20:42.961025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cece77f-3009-48da-b986-9e9273c97356 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:42.971566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cece77f-3009-48da-b986-9e9273c97356 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:43.028894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:43.029564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:43.029850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:43.030124 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] PATH_INFO: `/v3/projects/b52f26ab8b934756b0c74d681d08d04f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:43.031278 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52f26ab8b934756b0c74d681d08d04f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:43.033223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:43.157525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52f26ab8b934756b0c74d681d08d04f', 'request_id': 'req-0c22d40d-de8a-40e3-88a3-c7707af0417a'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:20:43.164748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52f26ab8b934756b0c74d681d08d04f', 'request_id': 'req-0c22d40d-de8a-40e3-88a3-c7707af0417a'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:20:43.234053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52f26ab8b934756b0c74d681d08d04f', 'request_id': 'req-0c22d40d-de8a-40e3-88a3-c7707af0417a'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:20:43.252604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] The token cache is being invalidate because project b52f26ab8b934756b0c74d681d08d04f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:20:43.252604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:20:43.252604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0c22d40d-de8a-40e3-88a3-c7707af0417a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:20:43.252604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1238/2488] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:20:42 2022] DELETE /identity/v3/projects/b52f26ab8b934756b0c74d681d08d04f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:20:43.256243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43df38-acd4-450c-b6df-10f3d5ed051e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:43.256555 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43df38-acd4-450c-b6df-10f3d5ed051e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:43.256834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43df38-acd4-450c-b6df-10f3d5ed051e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:43.295961 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:20:43.314604 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7c43df38-acd4-450c-b6df-10f3d5ed051e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:20:43.315830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7c43df38-acd4-450c-b6df-10f3d5ed051e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:43.373873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1251/2489] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:20:43.382669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfbc9e-bf46-4c47-9109-d71367974ba8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:43.382961 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfbc9e-bf46-4c47-9109-d71367974ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:43.383244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfbc9e-bf46-4c47-9109-d71367974ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:43.403124 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d0cfbc9e-bf46-4c47-9109-d71367974ba8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:20:43.404978 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d0cfbc9e-bf46-4c47-9109-d71367974ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:43.413390 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1239/2490] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:20:43.417828 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43df38-acd4-450c-b6df-10f3d5ed051e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:43.428870 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43df38-acd4-450c-b6df-10f3d5ed051e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:43.433510 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098a8fc-272d-4bcd-b829-f7c8e0faf3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:43.434145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098a8fc-272d-4bcd-b829-f7c8e0faf3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:43.434427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098a8fc-272d-4bcd-b829-f7c8e0faf3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:43.434702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098a8fc-272d-4bcd-b829-f7c8e0faf3d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:43.435397 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098a8fc-272d-4bcd-b829-f7c8e0faf3d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:20:43.435704 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098a8fc-272d-4bcd-b829-f7c8e0faf3d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:43.437401 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098a8fc-272d-4bcd-b829-f7c8e0faf3d9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:43.448781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1252/2491] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:20:43.453274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfbc9e-bf46-4c47-9109-d71367974ba8 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:43.464682 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfbc9e-bf46-4c47-9109-d71367974ba8 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:43.469955 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bea62-66f2-4a80-a587-c8b155b8ea1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:43.470575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bea62-66f2-4a80-a587-c8b155b8ea1d admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:43.470856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bea62-66f2-4a80-a587-c8b155b8ea1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:43.471134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bea62-66f2-4a80-a587-c8b155b8ea1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:43.471817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bea62-66f2-4a80-a587-c8b155b8ea1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:43.473749 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bea62-66f2-4a80-a587-c8b155b8ea1d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:43.560240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1240/2492] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:20:43 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:20:43.564366 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098a8fc-272d-4bcd-b829-f7c8e0faf3d9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:43.574762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098a8fc-272d-4bcd-b829-f7c8e0faf3d9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:43.632262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a35c05-02cd-4b09-afc0-96e4805e750d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:43.632879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35c05-02cd-4b09-afc0-96e4805e750d admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:43.633191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35c05-02cd-4b09-afc0-96e4805e750d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:43.633468 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35c05-02cd-4b09-afc0-96e4805e750d admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:43.634132 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a35c05-02cd-4b09-afc0-96e4805e750d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:43.636050 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a35c05-02cd-4b09-afc0-96e4805e750d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:43.712262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1253/2493] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:20:43 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:20:43.717255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bea62-66f2-4a80-a587-c8b155b8ea1d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:43.727664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bea62-66f2-4a80-a587-c8b155b8ea1d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:43.732285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdde97-fc70-4ef6-b275-73a20fa93e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:43.732929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdde97-fc70-4ef6-b275-73a20fa93e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:43.733300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdde97-fc70-4ef6-b275-73a20fa93e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:43.733582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdde97-fc70-4ef6-b275-73a20fa93e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:43.734336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdde97-fc70-4ef6-b275-73a20fa93e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:43.735920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdde97-fc70-4ef6-b275-73a20fa93e00 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:43.745728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1241/2494] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:20:43.749678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a35c05-02cd-4b09-afc0-96e4805e750d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:43.760309 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a35c05-02cd-4b09-afc0-96e4805e750d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:43.765916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1719fe-8948-42d2-8987-e66004c6ff82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:43.766541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1719fe-8948-42d2-8987-e66004c6ff82 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:43.766821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1719fe-8948-42d2-8987-e66004c6ff82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:43.767099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1719fe-8948-42d2-8987-e66004c6ff82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:43.767799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1719fe-8948-42d2-8987-e66004c6ff82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:43.769451 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1719fe-8948-42d2-8987-e66004c6ff82 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:43.780288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1254/2495] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:20:43.784543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdde97-fc70-4ef6-b275-73a20fa93e00 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:43.795251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdde97-fc70-4ef6-b275-73a20fa93e00 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:43.800393 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4fb08-ffad-42e0-909b-fd3ffe7f864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:43.800638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4fb08-ffad-42e0-909b-fd3ffe7f864c admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:43.800952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4fb08-ffad-42e0-909b-fd3ffe7f864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:43.801253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4fb08-ffad-42e0-909b-fd3ffe7f864c admin admin] PATH_INFO: `/v3/projects/104313ce72c94ea68c8cd3ee7f0cab2e/users/e2b1f1c4bb2747e0845cdc2991e53b09/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:43.816392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4fb08-ffad-42e0-909b-fd3ffe7f864c admin admin] RBAC: Authorizing `identity:create_grant(project_id=104313ce72c94ea68c8cd3ee7f0cab2e, user_id=e2b1f1c4bb2747e0845cdc2991e53b09, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:43.819249 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4fb08-ffad-42e0-909b-fd3ffe7f864c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:43.902729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1242/2496] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:20:43 2022] PUT /identity/v3/projects/104313ce72c94ea68c8cd3ee7f0cab2e/users/e2b1f1c4bb2747e0845cdc2991e53b09/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:20:43.908657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b24742-523e-46de-8618-57a50253bd1d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:43.909050 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b24742-523e-46de-8618-57a50253bd1d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:43.909432 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b24742-523e-46de-8618-57a50253bd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:43.929300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-20b24742-523e-46de-8618-57a50253bd1d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:20:43.930611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-20b24742-523e-46de-8618-57a50253bd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:44.001469 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1255/2497] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:20:44.013380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4fb08-ffad-42e0-909b-fd3ffe7f864c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:44.015121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e0a4fb08-ffad-42e0-909b-fd3ffe7f864c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:44.048727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:20:44.105968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4fb08-ffad-42e0-909b-fd3ffe7f864c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:44.109756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac86d2-a6a2-4ba3-8ce5-88111392f49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:44.110374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac86d2-a6a2-4ba3-8ce5-88111392f49f service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:44.113201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac86d2-a6a2-4ba3-8ce5-88111392f49f service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:44.113201 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac86d2-a6a2-4ba3-8ce5-88111392f49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:44.120303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac86d2-a6a2-4ba3-8ce5-88111392f49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:44.121768 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac86d2-a6a2-4ba3-8ce5-88111392f49f service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:44.131886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1243/2498] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:20:49.685520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b1ead-8858-4165-8ce7-51251306cb0a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:49.686050 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b1ead-8858-4165-8ce7-51251306cb0a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:49.686223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b1ead-8858-4165-8ce7-51251306cb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:49.695209 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:20:49.720080 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6c5b1ead-8858-4165-8ce7-51251306cb0a None None] MFA Rules not processed for user `e2b1f1c4bb2747e0845cdc2991e53b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:20:49.722222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6c5b1ead-8858-4165-8ce7-51251306cb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:49.807382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1256/2499] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:20:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:20:49.824096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac86d2-a6a2-4ba3-8ce5-88111392f49f service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:20:49.827143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-31ac86d2-a6a2-4ba3-8ce5-88111392f49f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:49.945488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac86d2-a6a2-4ba3-8ce5-88111392f49f service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:20:49.952311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6413d52-cd82-44da-bfef-4166c7e14b37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:20:49.953025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6413d52-cd82-44da-bfef-4166c7e14b37 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:49.953384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6413d52-cd82-44da-bfef-4166c7e14b37 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:49.953732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6413d52-cd82-44da-bfef-4166c7e14b37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:49.964780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6413d52-cd82-44da-bfef-4166c7e14b37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:20:49.967949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6413d52-cd82-44da-bfef-4166c7e14b37 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:20:49.977420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1244/2500] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:20:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:20:52.920987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf58810-4593-4d95-aa15-6d6a233645b9 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:20:52.921531 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf58810-4593-4d95-aa15-6d6a233645b9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:20:52.921691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf58810-4593-4d95-aa15-6d6a233645b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:20:52.941856 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-dcf58810-4593-4d95-aa15-6d6a233645b9 None None] MFA Rules not processed for user `e2b1f1c4bb2747e0845cdc2991e53b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:20:52.943200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dcf58810-4593-4d95-aa15-6d6a233645b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:20:52.950879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1257/2501] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:20:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:20:59.972071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=69 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:21:02.936131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=33 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:21:06.373889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e997a8-e385-402d-b2da-3cff7eb8ddc3 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:21:06.374405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e997a8-e385-402d-b2da-3cff7eb8ddc3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:21:06.374690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e997a8-e385-402d-b2da-3cff7eb8ddc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:21:06.393494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-63e997a8-e385-402d-b2da-3cff7eb8ddc3 None None] MFA Rules not processed for user `30f0002d2a5a4950b2f1aa771f33bbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:21:06.394709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-63e997a8-e385-402d-b2da-3cff7eb8ddc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:21:06.460875 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1245/2502] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:21:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:21:16.454053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=25 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:22:12.349088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05b14b-22de-49af-bc0a-4a7ece2d2605 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:12.349579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05b14b-22de-49af-bc0a-4a7ece2d2605 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:12.349793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05b14b-22de-49af-bc0a-4a7ece2d2605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:12.379858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6c05b14b-22de-49af-bc0a-4a7ece2d2605 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:22:12.381097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6c05b14b-22de-49af-bc0a-4a7ece2d2605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:12.440344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1258/2503] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:22:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:22:12.578350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e997a8-e385-402d-b2da-3cff7eb8ddc3 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:22:12.579758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-63e997a8-e385-402d-b2da-3cff7eb8ddc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:12.649784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e997a8-e385-402d-b2da-3cff7eb8ddc3 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:22:12.654831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181163e-4edc-4e2c-a586-1739eeb8e505 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:22:12.655450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181163e-4edc-4e2c-a586-1739eeb8e505 service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:12.655732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181163e-4edc-4e2c-a586-1739eeb8e505 service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:12.656009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181163e-4edc-4e2c-a586-1739eeb8e505 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:12.664663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181163e-4edc-4e2c-a586-1739eeb8e505 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:22:12.666280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181163e-4edc-4e2c-a586-1739eeb8e505 service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:22:12.674347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1246/2504] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:22:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:22:22.433406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=29 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:22:22.671077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=32 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:22:27.788782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05b14b-22de-49af-bc0a-4a7ece2d2605 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:22:27.798376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05b14b-22de-49af-bc0a-4a7ece2d2605 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:22:27.802377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec47abc-36e9-4aa1-b48c-75115994a32c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:22:27.803007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec47abc-36e9-4aa1-b48c-75115994a32c service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:27.803292 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec47abc-36e9-4aa1-b48c-75115994a32c service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:27.803571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec47abc-36e9-4aa1-b48c-75115994a32c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:27.805410 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0ec47abc-36e9-4aa1-b48c-75115994a32c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:27.819930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec47abc-36e9-4aa1-b48c-75115994a32c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:22:27.821417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec47abc-36e9-4aa1-b48c-75115994a32c service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:22:27.833848 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1259/2505] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:22:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:22:29.089181 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181163e-4edc-4e2c-a586-1739eeb8e505 service swift] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:22:29.099427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181163e-4edc-4e2c-a586-1739eeb8e505 service swift] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:22:29.103067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115f439-80cf-4e0a-9a9e-101488a9f358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:22:29.103673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f439-80cf-4e0a-9a9e-101488a9f358 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:29.103974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f439-80cf-4e0a-9a9e-101488a9f358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:29.104287 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f439-80cf-4e0a-9a9e-101488a9f358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:29.106090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f115f439-80cf-4e0a-9a9e-101488a9f358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:29.119969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115f439-80cf-4e0a-9a9e-101488a9f358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:22:29.121435 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115f439-80cf-4e0a-9a9e-101488a9f358 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:22:29.132916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1247/2506] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:22:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:22:32.787392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92082582-41b2-41e1-bf94-665067115b79 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:32.787957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92082582-41b2-41e1-bf94-665067115b79 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:32.787957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92082582-41b2-41e1-bf94-665067115b79 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:32.789415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1260/2507] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:22:32.795184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6a64-30e2-4985-b725-bb28f64ea7ea None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:32.795643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6a64-30e2-4985-b725-bb28f64ea7ea None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:32.795879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6a64-30e2-4985-b725-bb28f64ea7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:32.838957 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0d6d6a64-30e2-4985-b725-bb28f64ea7ea None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:22:32.840354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0d6d6a64-30e2-4985-b725-bb28f64ea7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:32.850968 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1248/2508] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:22:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:22:32.862363 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92082582-41b2-41e1-bf94-665067115b79 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:22:32.864094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-92082582-41b2-41e1-bf94-665067115b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:32.922929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92082582-41b2-41e1-bf94-665067115b79 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:22:32.928036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b0eca-80b2-4638-b339-3beedee959b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47f1552e73648ceb8a421c0543c2974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:22:32.928595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b0eca-80b2-4638-b339-3beedee959b7 service glance] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:32.928879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b0eca-80b2-4638-b339-3beedee959b7 service glance] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:32.929186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b0eca-80b2-4638-b339-3beedee959b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:32.939649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b0eca-80b2-4638-b339-3beedee959b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:22:32.941601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b0eca-80b2-4638-b339-3beedee959b7 service glance] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:22:32.953158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1261/2509] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:22:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:22:33.152232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51a8aa-a7bb-4a08-a783-27564958d2ce None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:33.152513 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51a8aa-a7bb-4a08-a783-27564958d2ce None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:33.152793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51a8aa-a7bb-4a08-a783-27564958d2ce None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:33.154393 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1249/2510] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:22:33.159629 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298480c3-8272-4a3d-a771-f99fb8e12923 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:33.160362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298480c3-8272-4a3d-a771-f99fb8e12923 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:33.160641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298480c3-8272-4a3d-a771-f99fb8e12923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:33.201419 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-298480c3-8272-4a3d-a771-f99fb8e12923 None None] MFA Rules not processed for user `b038336582a0435dbd47ad2c09cd23e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:22:33.202689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-298480c3-8272-4a3d-a771-f99fb8e12923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:33.210128 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1262/2511] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:22:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:22:33.222251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51a8aa-a7bb-4a08-a783-27564958d2ce None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:22:33.233554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51a8aa-a7bb-4a08-a783-27564958d2ce None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:22:33.237189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41006c-dca9-43d1-a72b-00157d23a9b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:22:33.237959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41006c-dca9-43d1-a72b-00157d23a9b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:33.238259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41006c-dca9-43d1-a72b-00157d23a9b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:33.238538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41006c-dca9-43d1-a72b-00157d23a9b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:33.246023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41006c-dca9-43d1-a72b-00157d23a9b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:22:33.251294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41006c-dca9-43d1-a72b-00157d23a9b3 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:22:33.266133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1250/2512] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:22:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:22:33.345511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f9ff-e34f-4e2d-8e93-3b3518c3ba91 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:33.345806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f9ff-e34f-4e2d-8e93-3b3518c3ba91 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:33.346091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f9ff-e34f-4e2d-8e93-3b3518c3ba91 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:33.347544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1263/2513] 10.222.0.86 () {54 vars in 921 bytes} [Thu Jul 28 18:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 18:22:33.353105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398bc3-b1e0-48d7-8c24-6d5418b1a05f None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:33.353399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398bc3-b1e0-48d7-8c24-6d5418b1a05f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:33.353678 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398bc3-b1e0-48d7-8c24-6d5418b1a05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:33.374204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-91398bc3-b1e0-48d7-8c24-6d5418b1a05f None None] MFA Rules not processed for user `7d7bcb4e7c434622b650cdfc500a9680`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:22:33.375435 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-91398bc3-b1e0-48d7-8c24-6d5418b1a05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:33.386098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1251/2514] 10.222.0.86 () {58 vars in 1018 bytes} [Thu Jul 28 18:22:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:22:33.396438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8f9ff-e34f-4e2d-8e93-3b3518c3ba91 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:22:33.407490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8f9ff-e34f-4e2d-8e93-3b3518c3ba91 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:22:33.411158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d0bcc-8099-4f87-8555-1f92470d049d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:22:33.411891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d0bcc-8099-4f87-8555-1f92470d049d service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:33.412176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d0bcc-8099-4f87-8555-1f92470d049d service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:33.412456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d0bcc-8099-4f87-8555-1f92470d049d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:33.428936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d0bcc-8099-4f87-8555-1f92470d049d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:22:33.428936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d0bcc-8099-4f87-8555-1f92470d049d service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:22:33.437480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1264/2515] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:22:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:22:35.908589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d679e8-3a35-481b-a098-d4f5ad257317 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:35.909104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d679e8-3a35-481b-a098-d4f5ad257317 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:35.909212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d679e8-3a35-481b-a098-d4f5ad257317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:35.929993 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a7d679e8-3a35-481b-a098-d4f5ad257317 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:22:35.931260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a7d679e8-3a35-481b-a098-d4f5ad257317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:35.938707 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1252/2516] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:22:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:22:35.951059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d0bcc-8099-4f87-8555-1f92470d049d service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:22:35.963673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d0bcc-8099-4f87-8555-1f92470d049d service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:22:35.969442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d8acb-aecf-4a83-8db9-5edfad0cb5de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:22:35.970087 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d8acb-aecf-4a83-8db9-5edfad0cb5de service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:35.970370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d8acb-aecf-4a83-8db9-5edfad0cb5de service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:35.970644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d8acb-aecf-4a83-8db9-5edfad0cb5de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:35.979643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d8acb-aecf-4a83-8db9-5edfad0cb5de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:22:35.981480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d8acb-aecf-4a83-8db9-5edfad0cb5de service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:22:35.990623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1265/2517] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:22:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:22:42.278937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d679e8-3a35-481b-a098-d4f5ad257317 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:22:42.280660 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a7d679e8-3a35-481b-a098-d4f5ad257317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:42.298942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d679e8-3a35-481b-a098-d4f5ad257317 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:22:42.306486 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86901f-537b-4d38-801a-0fba7bccc462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:22:42.307233 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86901f-537b-4d38-801a-0fba7bccc462 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:42.307519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86901f-537b-4d38-801a-0fba7bccc462 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:42.307794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86901f-537b-4d38-801a-0fba7bccc462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:42.319476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7b86901f-537b-4d38-801a-0fba7bccc462 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:42.422264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86901f-537b-4d38-801a-0fba7bccc462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:22:42.424132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86901f-537b-4d38-801a-0fba7bccc462 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:22:42.447487 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1253/2518] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:22:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:22:45.987071 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=53 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:22:52.444205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=68 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:22:59.569484 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6d921-a225-4da3-a296-48c32f320386 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:59.569991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6d921-a225-4da3-a296-48c32f320386 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:59.569991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6d921-a225-4da3-a296-48c32f320386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:59.618365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a0f6d921-a225-4da3-a296-48c32f320386 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:22:59.619565 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a0f6d921-a225-4da3-a296-48c32f320386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:22:59.635522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1266/2519] 10.222.0.86 () {58 vars in 967 bytes} [Thu Jul 28 18:22:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:22:59.647638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86901f-537b-4d38-801a-0fba7bccc462 service nova] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:22:59.668854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86901f-537b-4d38-801a-0fba7bccc462 service nova] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:22:59.673936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72b529-f9ae-42c8-8317-e8df568ab426 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:22:59.674550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72b529-f9ae-42c8-8317-e8df568ab426 service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:22:59.674832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72b529-f9ae-42c8-8317-e8df568ab426 service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:22:59.675106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72b529-f9ae-42c8-8317-e8df568ab426 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:22:59.683882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72b529-f9ae-42c8-8317-e8df568ab426 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:22:59.685520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72b529-f9ae-42c8-8317-e8df568ab426 service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:22:59.693562 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1254/2520] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:22:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:23:09.632759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=18 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:09.690308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=10 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:12.311217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259b144-fbca-407a-933f-c428e263efb6 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:12.311729 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259b144-fbca-407a-933f-c428e263efb6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:12.311827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259b144-fbca-407a-933f-c428e263efb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:12.331795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0259b144-fbca-407a-933f-c428e263efb6 None None] MFA Rules not processed for user `e2b1f1c4bb2747e0845cdc2991e53b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:23:12.333030 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0259b144-fbca-407a-933f-c428e263efb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:12.340822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1267/2521] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:23:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:23:12.652420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72b529-f9ae-42c8-8317-e8df568ab426 service swift] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:12.661640 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72b529-f9ae-42c8-8317-e8df568ab426 service swift] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:12.665363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef19e1-c4ce-4878-9aa9-4e69eeec8a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:12.665965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef19e1-c4ce-4878-9aa9-4e69eeec8a8b service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:12.666248 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef19e1-c4ce-4878-9aa9-4e69eeec8a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:12.666523 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef19e1-c4ce-4878-9aa9-4e69eeec8a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:12.668262 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-aeef19e1-c4ce-4878-9aa9-4e69eeec8a8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:12.678155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef19e1-c4ce-4878-9aa9-4e69eeec8a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:12.679593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef19e1-c4ce-4878-9aa9-4e69eeec8a8b service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:12.687501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1255/2522] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:23:13.105335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259b144-fbca-407a-933f-c428e263efb6 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:13.106710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0259b144-fbca-407a-933f-c428e263efb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:13.178969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259b144-fbca-407a-933f-c428e263efb6 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:13.183966 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993b628-2c1f-4441-a0cd-599ec6505a15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:13.184591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993b628-2c1f-4441-a0cd-599ec6505a15 service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:13.184883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993b628-2c1f-4441-a0cd-599ec6505a15 service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:13.185193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993b628-2c1f-4441-a0cd-599ec6505a15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:13.186946 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7993b628-2c1f-4441-a0cd-599ec6505a15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:13.198527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993b628-2c1f-4441-a0cd-599ec6505a15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:13.200291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993b628-2c1f-4441-a0cd-599ec6505a15 service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:13.210497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1268/2523] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:23:17.104586 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef19e1-c4ce-4878-9aa9-4e69eeec8a8b service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:17.113318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef19e1-c4ce-4878-9aa9-4e69eeec8a8b service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:17.117186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6f809-14b3-4435-8ff0-d4ae54cd761a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:17.121895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6f809-14b3-4435-8ff0-d4ae54cd761a service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:17.121895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6f809-14b3-4435-8ff0-d4ae54cd761a service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:17.121895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6f809-14b3-4435-8ff0-d4ae54cd761a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:17.121895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-26e6f809-14b3-4435-8ff0-d4ae54cd761a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:17.130535 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6f809-14b3-4435-8ff0-d4ae54cd761a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:17.132007 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6f809-14b3-4435-8ff0-d4ae54cd761a service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:17.140672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1256/2524] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:23:18.016277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993b628-2c1f-4441-a0cd-599ec6505a15 service placement] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:18.030244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993b628-2c1f-4441-a0cd-599ec6505a15 service placement] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:18.036179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a559-8149-439d-b85a-5b2f2c944e84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:18.036908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a559-8149-439d-b85a-5b2f2c944e84 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:18.037553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a559-8149-439d-b85a-5b2f2c944e84 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:18.037553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a559-8149-439d-b85a-5b2f2c944e84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:18.039522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3d86a559-8149-439d-b85a-5b2f2c944e84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:18.057578 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a559-8149-439d-b85a-5b2f2c944e84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:18.059363 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a559-8149-439d-b85a-5b2f2c944e84 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:18.074192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1269/2525] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:23:24.048491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6f809-14b3-4435-8ff0-d4ae54cd761a service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:24.049971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-26e6f809-14b3-4435-8ff0-d4ae54cd761a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:24.060938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6f809-14b3-4435-8ff0-d4ae54cd761a service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:24.064592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7367d9b-9926-4557-b992-3ea4c7149eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:24.065247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7367d9b-9926-4557-b992-3ea4c7149eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:24.065526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7367d9b-9926-4557-b992-3ea4c7149eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:24.065930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7367d9b-9926-4557-b992-3ea4c7149eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:24.067519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f7367d9b-9926-4557-b992-3ea4c7149eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:24.132876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7367d9b-9926-4557-b992-3ea4c7149eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:24.134338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7367d9b-9926-4557-b992-3ea4c7149eb8 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:24.143525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1257/2526] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:23:25.185215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a559-8149-439d-b85a-5b2f2c944e84 service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:25.196947 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a559-8149-439d-b85a-5b2f2c944e84 service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:25.203440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cedd7-c826-469a-b42a-6999e772bd56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:25.204190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cedd7-c826-469a-b42a-6999e772bd56 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:25.204538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cedd7-c826-469a-b42a-6999e772bd56 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:25.204905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cedd7-c826-469a-b42a-6999e772bd56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:25.206912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-568cedd7-c826-469a-b42a-6999e772bd56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:25.222101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cedd7-c826-469a-b42a-6999e772bd56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:25.226838 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cedd7-c826-469a-b42a-6999e772bd56 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:25.240478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1270/2527] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:23:32.990285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7367d9b-9926-4557-b992-3ea4c7149eb8 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:32.999134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7367d9b-9926-4557-b992-3ea4c7149eb8 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:33.002800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7fba0-cb7e-4ab1-9062-61de2a9146f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:33.003442 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7fba0-cb7e-4ab1-9062-61de2a9146f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:33.003695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7fba0-cb7e-4ab1-9062-61de2a9146f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:33.003969 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7fba0-cb7e-4ab1-9062-61de2a9146f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:33.006463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-bcd7fba0-cb7e-4ab1-9062-61de2a9146f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:33.018226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7fba0-cb7e-4ab1-9062-61de2a9146f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:33.019639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7fba0-cb7e-4ab1-9062-61de2a9146f9 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:33.029733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1258/2528] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:23:35.234606 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=47 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:36.244890 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cedd7-c826-469a-b42a-6999e772bd56 service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:36.259452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cedd7-c826-469a-b42a-6999e772bd56 service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:36.263423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a5117-30fc-45a0-8f5f-7bc7a75f3a06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:36.264058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a5117-30fc-45a0-8f5f-7bc7a75f3a06 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:36.264674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a5117-30fc-45a0-8f5f-7bc7a75f3a06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:36.264988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a5117-30fc-45a0-8f5f-7bc7a75f3a06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:36.266868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c12a5117-30fc-45a0-8f5f-7bc7a75f3a06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:36.280291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a5117-30fc-45a0-8f5f-7bc7a75f3a06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:36.281815 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a5117-30fc-45a0-8f5f-7bc7a75f3a06 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:36.294614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1271/2529] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:23:36.673873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7fba0-cb7e-4ab1-9062-61de2a9146f9 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:36.676846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-bcd7fba0-cb7e-4ab1-9062-61de2a9146f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:36.696281 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7fba0-cb7e-4ab1-9062-61de2a9146f9 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:36.702192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338590e-9235-452b-93d9-aaee67bdd83d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:36.703142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338590e-9235-452b-93d9-aaee67bdd83d service placement] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:36.703651 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338590e-9235-452b-93d9-aaee67bdd83d service placement] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:36.704146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338590e-9235-452b-93d9-aaee67bdd83d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:36.706198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1338590e-9235-452b-93d9-aaee67bdd83d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:36.720191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338590e-9235-452b-93d9-aaee67bdd83d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:36.721807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338590e-9235-452b-93d9-aaee67bdd83d service placement] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:36.732192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1259/2530] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:23:41.386169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a5117-30fc-45a0-8f5f-7bc7a75f3a06 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:41.388930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c12a5117-30fc-45a0-8f5f-7bc7a75f3a06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:41.418790 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a5117-30fc-45a0-8f5f-7bc7a75f3a06 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:41.429685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:41.430411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:41.431184 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:41.431689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] PATH_INFO: `/v3/users/b9874df18af245688db1137b6f1b0bff` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:41.448602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9874df18af245688db1137b6f1b0bff)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:41.450602 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:41.480531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338590e-9235-452b-93d9-aaee67bdd83d service placement] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:41.486152 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=53 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:41.492866 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338590e-9235-452b-93d9-aaee67bdd83d service placement] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:41.498372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162508b6-fa05-4a19-b86b-f06964a1de95 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:41.499205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162508b6-fa05-4a19-b86b-f06964a1de95 service placement] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:41.500000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162508b6-fa05-4a19-b86b-f06964a1de95 service placement] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:41.500293 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162508b6-fa05-4a19-b86b-f06964a1de95 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:41.502389 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-162508b6-fa05-4a19-b86b-f06964a1de95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:41.517425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162508b6-fa05-4a19-b86b-f06964a1de95 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:41.519470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162508b6-fa05-4a19-b86b-f06964a1de95 service placement] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:41.531422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1260/2531] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:23:41.619540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162508b6-fa05-4a19-b86b-f06964a1de95 service placement] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:41.619540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-162508b6-fa05-4a19-b86b-f06964a1de95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:41.646673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162508b6-fa05-4a19-b86b-f06964a1de95 service placement] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:41.649841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:41.650892 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:41.651449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:41.652466 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] PATH_INFO: `/v3/users/83ba025a383c433b900a3f4ae64959b3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:41.654135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ba025a383c433b900a3f4ae64959b3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:41.654912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9874df18af245688db1137b6f1b0bff'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:41.655462 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9874df18af245688db1137b6f1b0bff'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:41.657649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:41.664672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9874df18af245688db1137b6f1b0bff'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:41.754943 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9874df18af245688db1137b6f1b0bff'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:41.756166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9874df18af245688db1137b6f1b0bff'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:41.774661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1272/2532] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:23:41 2022] DELETE /identity/v3/users/b9874df18af245688db1137b6f1b0bff => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:23:41.891102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ba025a383c433b900a3f4ae64959b3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:41.898296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ba025a383c433b900a3f4ae64959b3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:41.990159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ba025a383c433b900a3f4ae64959b3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:41.991090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ba025a383c433b900a3f4ae64959b3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.003992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ba025a383c433b900a3f4ae64959b3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.010542 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1261/2533] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:23:41 2022] DELETE /identity/v3/users/83ba025a383c433b900a3f4ae64959b3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:23:42.028872 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:42.030388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:42.108112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee98168-c166-4c03-899a-cf58476e5c39 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:42.158573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:42.165461 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:42.168022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:42.168022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:42.168022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] PATH_INFO: `/v3/projects/27b91f25d2be408ba9e8d667a574a7a9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:42.177889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b91f25d2be408ba9e8d667a574a7a9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:42.179810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:42.322215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:42.322215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:42.331731 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b91f25d2be408ba9e8d667a574a7a9', 'request_id': 'req-4fb9f75f-c27f-4b40-9930-ec6d19b669be'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.339641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b91f25d2be408ba9e8d667a574a7a9', 'request_id': 'req-4fb9f75f-c27f-4b40-9930-ec6d19b669be'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.344901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59019dfb-2d70-40f9-89f6-4a2ca0bb6e1f admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:42.347463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b91f25d2be408ba9e8d667a574a7a9', 'request_id': 'req-4fb9f75f-c27f-4b40-9930-ec6d19b669be'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.405989 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:42.406640 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:42.406927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:42.407208 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] PATH_INFO: `/v3/projects/bf347b1959b247bcbe003ac3662cb239` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:42.408399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf347b1959b247bcbe003ac3662cb239)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:42.410344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:42.451499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] The token cache is being invalidate because project 27b91f25d2be408ba9e8d667a574a7a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:23:42.452004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.452418 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fb9f75f-c27f-4b40-9930-ec6d19b669be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.454028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1273/2534] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:23:42 2022] DELETE /identity/v3/projects/27b91f25d2be408ba9e8d667a574a7a9 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:23:42.458965 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166e8bc-cd18-49c2-8408-7b2e3426e04d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:42.459250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166e8bc-cd18-49c2-8408-7b2e3426e04d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:42.459862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166e8bc-cd18-49c2-8408-7b2e3426e04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:42.482150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8166e8bc-cd18-49c2-8408-7b2e3426e04d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:23:42.483374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8166e8bc-cd18-49c2-8408-7b2e3426e04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:42.491512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1274/2535] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:23:42.500508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461459-6db9-45f7-9036-d729eb2c1d66 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:42.500802 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461459-6db9-45f7-9036-d729eb2c1d66 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:42.501121 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461459-6db9-45f7-9036-d729eb2c1d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:42.519743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-07461459-6db9-45f7-9036-d729eb2c1d66 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:23:42.521010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-07461459-6db9-45f7-9036-d729eb2c1d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:42.529245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1275/2536] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:23:42.533374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07461459-6db9-45f7-9036-d729eb2c1d66 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:42.540385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf347b1959b247bcbe003ac3662cb239', 'request_id': 'req-e7f822f4-cb7b-4791-8a70-660a65bbfdca'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.542727 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07461459-6db9-45f7-9036-d729eb2c1d66 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:42.546103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf347b1959b247bcbe003ac3662cb239', 'request_id': 'req-e7f822f4-cb7b-4791-8a70-660a65bbfdca'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.549980 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:42.599480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f16d21-fcb1-453e-a497-0f90d8a0124a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:42.600104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f16d21-fcb1-453e-a497-0f90d8a0124a admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:42.600394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f16d21-fcb1-453e-a497-0f90d8a0124a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:42.600671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f16d21-fcb1-453e-a497-0f90d8a0124a admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:42.601011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf347b1959b247bcbe003ac3662cb239', 'request_id': 'req-e7f822f4-cb7b-4791-8a70-660a65bbfdca'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.601397 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f16d21-fcb1-453e-a497-0f90d8a0124a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:23:42.601710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f16d21-fcb1-453e-a497-0f90d8a0124a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:42.603313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f16d21-fcb1-453e-a497-0f90d8a0124a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:42.614859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1276/2537] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:23:42.615343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] The token cache is being invalidate because project bf347b1959b247bcbe003ac3662cb239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:23:42.615647 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.616335 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.617945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1262/2538] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:23:42 2022] DELETE /identity/v3/projects/bf347b1959b247bcbe003ac3662cb239 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:23:42.619249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f16d21-fcb1-453e-a497-0f90d8a0124a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:42.621246 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-56f16d21-fcb1-453e-a497-0f90d8a0124a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:42.621644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:42.622949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:42.635960 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f822f4-cb7b-4791-8a70-660a65bbfdca admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:42.637635 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f16d21-fcb1-453e-a497-0f90d8a0124a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:42.642061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab29358-1e8b-4ab5-902e-5240a7087bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:42.642734 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab29358-1e8b-4ab5-902e-5240a7087bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:42.643016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab29358-1e8b-4ab5-902e-5240a7087bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:42.643293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab29358-1e8b-4ab5-902e-5240a7087bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:42.643969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab29358-1e8b-4ab5-902e-5240a7087bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:42.644913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:42.646097 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:42.646392 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:42.646551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab29358-1e8b-4ab5-902e-5240a7087bdd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:42.647274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] PATH_INFO: `/v3/users/9ea265df98a24fe69d26603db19315b1` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:42.648482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea265df98a24fe69d26603db19315b1)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:42.650474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:42.745850 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1277/2539] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:23:42 2022] POST /identity/v3/projects => generated 407 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:23:42.750086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab29358-1e8b-4ab5-902e-5240a7087bdd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:42.783063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:42.816942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab29358-1e8b-4ab5-902e-5240a7087bdd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:42.823280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea265df98a24fe69d26603db19315b1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.823429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a7ea5-e75f-4cbb-be85-667ba103edff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:42.824092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a7ea5-e75f-4cbb-be85-667ba103edff admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:42.824376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a7ea5-e75f-4cbb-be85-667ba103edff admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:42.824651 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a7ea5-e75f-4cbb-be85-667ba103edff admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:42.825364 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a7ea5-e75f-4cbb-be85-667ba103edff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:42.827553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a7ea5-e75f-4cbb-be85-667ba103edff admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:42.833704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea265df98a24fe69d26603db19315b1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.989481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea265df98a24fe69d26603db19315b1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:42.990089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea265df98a24fe69d26603db19315b1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:43.002831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea265df98a24fe69d26603db19315b1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:43.009455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1263/2540] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:23:42 2022] DELETE /identity/v3/users/9ea265df98a24fe69d26603db19315b1 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:23:43.016816 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1278/2541] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:23:42 2022] POST /identity/v3/users => generated 304 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:23:43.018151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:43.019516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:43.091338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e1785-93b7-4ba5-98fd-2f2c6e5994ea admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:43.096148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96431996-da7f-48f4-8a90-240195f63a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:43.096770 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96431996-da7f-48f4-8a90-240195f63a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:43.097088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96431996-da7f-48f4-8a90-240195f63a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:43.097377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96431996-da7f-48f4-8a90-240195f63a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:43.098083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96431996-da7f-48f4-8a90-240195f63a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:43.099741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96431996-da7f-48f4-8a90-240195f63a33 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:43.109719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1264/2542] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:23:43.113775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a7ea5-e75f-4cbb-be85-667ba103edff admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:43.124058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a7ea5-e75f-4cbb-be85-667ba103edff admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:43.128889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966fd32-cfd0-45d7-8ea6-e3d788384810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:43.129535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966fd32-cfd0-45d7-8ea6-e3d788384810 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:43.129823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966fd32-cfd0-45d7-8ea6-e3d788384810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:43.130716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966fd32-cfd0-45d7-8ea6-e3d788384810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:43.131428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966fd32-cfd0-45d7-8ea6-e3d788384810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:43.133058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966fd32-cfd0-45d7-8ea6-e3d788384810 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:43.143127 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1279/2543] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:23:43.147983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96431996-da7f-48f4-8a90-240195f63a33 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:43.158395 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96431996-da7f-48f4-8a90-240195f63a33 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:43.163046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fd0aa-01b4-46ae-8e25-107ca7cb50e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:43.163708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fd0aa-01b4-46ae-8e25-107ca7cb50e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:43.163990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fd0aa-01b4-46ae-8e25-107ca7cb50e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:43.164264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fd0aa-01b4-46ae-8e25-107ca7cb50e6 admin admin] PATH_INFO: `/v3/projects/2b4e63e7c1024cf5a76d41875aad53b6/users/b263ac3c4e3f446c961ba8d715e369da/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:43.179625 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891fd0aa-01b4-46ae-8e25-107ca7cb50e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4e63e7c1024cf5a76d41875aad53b6, user_id=b263ac3c4e3f446c961ba8d715e369da, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:43.182348 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891fd0aa-01b4-46ae-8e25-107ca7cb50e6 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:43.304940 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966fd32-cfd0-45d7-8ea6-e3d788384810 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:43.306696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0966fd32-cfd0-45d7-8ea6-e3d788384810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:43.320649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966fd32-cfd0-45d7-8ea6-e3d788384810 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:43.336847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1265/2544] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:23:43 2022] PUT /identity/v3/projects/2b4e63e7c1024cf5a76d41875aad53b6/users/b263ac3c4e3f446c961ba8d715e369da/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:23:43.338381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c500e06-ee20-446c-ac6e-4fd0bd4505c8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:43.338675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c500e06-ee20-446c-ac6e-4fd0bd4505c8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:43.338956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c500e06-ee20-446c-ac6e-4fd0bd4505c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:43.367127 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5c500e06-ee20-446c-ac6e-4fd0bd4505c8 None None] MFA Rules not processed for user `b263ac3c4e3f446c961ba8d715e369da`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:23:43.368409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5c500e06-ee20-446c-ac6e-4fd0bd4505c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:43.382717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:43.383466 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:43.383783 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:43.384060 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] PATH_INFO: `/v3/projects/40267a64c0a943fea33830788584f799` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:43.385287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=40267a64c0a943fea33830788584f799)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:43.387189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:43.394271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:43.435320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1266/2545] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:23:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:23:43.439887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c500e06-ee20-446c-ac6e-4fd0bd4505c8 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:43.452061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c500e06-ee20-446c-ac6e-4fd0bd4505c8 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:43.456853 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afe8a7-a993-4d64-bf4d-c87c9ec61077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:43.457508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afe8a7-a993-4d64-bf4d-c87c9ec61077 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:43.457796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afe8a7-a993-4d64-bf4d-c87c9ec61077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:43.458112 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afe8a7-a993-4d64-bf4d-c87c9ec61077 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:43.458830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afe8a7-a993-4d64-bf4d-c87c9ec61077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:43.460801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afe8a7-a993-4d64-bf4d-c87c9ec61077 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:43.500322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40267a64c0a943fea33830788584f799', 'request_id': 'req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:43.506821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40267a64c0a943fea33830788584f799', 'request_id': 'req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:43.516473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40267a64c0a943fea33830788584f799', 'request_id': 'req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:43.540060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1267/2546] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:23:43 2022] POST /identity/v3/projects => generated 409 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:23:43.543420 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afe8a7-a993-4d64-bf4d-c87c9ec61077 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:43.605307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afe8a7-a993-4d64-bf4d-c87c9ec61077 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:43.610212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2183c0-6485-4b03-911f-431f35b43850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:43.610936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2183c0-6485-4b03-911f-431f35b43850 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:43.611220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2183c0-6485-4b03-911f-431f35b43850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:43.611489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2183c0-6485-4b03-911f-431f35b43850 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:43.611971 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] The token cache is being invalidate because project 40267a64c0a943fea33830788584f799 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:23:43.612153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2183c0-6485-4b03-911f-431f35b43850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:43.612286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:43.612952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:43.614198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2183c0-6485-4b03-911f-431f35b43850 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:43.615029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1280/2547] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:23:43 2022] DELETE /identity/v3/projects/40267a64c0a943fea33830788584f799 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:23:43.682251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1268/2548] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:23:43 2022] POST /identity/v3/users => generated 311 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:23:43.686502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:43.688214 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:43.733682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:43.760090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21c897-fd89-4aba-940f-3d3b7fbe78fc admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:43.765553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b993b-fb72-4b84-b293-07687709c726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:43.766202 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b993b-fb72-4b84-b293-07687709c726 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:43.766505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b993b-fb72-4b84-b293-07687709c726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:43.766769 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b993b-fb72-4b84-b293-07687709c726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:43.767492 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b993b-fb72-4b84-b293-07687709c726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:43.769161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b993b-fb72-4b84-b293-07687709c726 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:43.783507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1281/2549] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:23:43.784878 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2183c0-6485-4b03-911f-431f35b43850 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:43.796429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2183c0-6485-4b03-911f-431f35b43850 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:43.801579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc818fe-d34b-4919-9dc1-a907e32b8f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:43.802937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc818fe-d34b-4919-9dc1-a907e32b8f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:43.803232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc818fe-d34b-4919-9dc1-a907e32b8f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:43.803517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc818fe-d34b-4919-9dc1-a907e32b8f83 admin admin] PATH_INFO: `/v3/projects/fd883effc4f942a09ac4aeb6542319fa/users/fe3669f1e03447f7a5793e454101cde2/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:43.821485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc818fe-d34b-4919-9dc1-a907e32b8f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd883effc4f942a09ac4aeb6542319fa, user_id=fe3669f1e03447f7a5793e454101cde2, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:43.824353 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc818fe-d34b-4919-9dc1-a907e32b8f83 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:43.909966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1269/2550] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:23:43 2022] PUT /identity/v3/projects/fd883effc4f942a09ac4aeb6542319fa/users/fe3669f1e03447f7a5793e454101cde2/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:23:43.914685 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b993b-fb72-4b84-b293-07687709c726 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:43.925114 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b993b-fb72-4b84-b293-07687709c726 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:43.982235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81ff6c-0881-46d4-b870-1e1604d523ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:43.982864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81ff6c-0881-46d4-b870-1e1604d523ae admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:43.983323 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81ff6c-0881-46d4-b870-1e1604d523ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:43.983459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81ff6c-0881-46d4-b870-1e1604d523ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:43.984177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81ff6c-0881-46d4-b870-1e1604d523ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:43.985866 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81ff6c-0881-46d4-b870-1e1604d523ae admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:43.996732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1282/2551] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:23:44.001174 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc818fe-d34b-4919-9dc1-a907e32b8f83 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:44.011705 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc818fe-d34b-4919-9dc1-a907e32b8f83 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:44.016436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa071fdb-6e18-4738-bb0e-b6ea8ae2d3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:44.017051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa071fdb-6e18-4738-bb0e-b6ea8ae2d3b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:44.017341 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa071fdb-6e18-4738-bb0e-b6ea8ae2d3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:44.017621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa071fdb-6e18-4738-bb0e-b6ea8ae2d3b2 admin admin] PATH_INFO: `/v3/projects/fd883effc4f942a09ac4aeb6542319fa/users/fe3669f1e03447f7a5793e454101cde2/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:44.019598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa071fdb-6e18-4738-bb0e-b6ea8ae2d3b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd883effc4f942a09ac4aeb6542319fa, user_id=fe3669f1e03447f7a5793e454101cde2, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:44.022284 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa071fdb-6e18-4738-bb0e-b6ea8ae2d3b2 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:44.032443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1270/2552] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:23:44 2022] PUT /identity/v3/projects/fd883effc4f942a09ac4aeb6542319fa/users/fe3669f1e03447f7a5793e454101cde2/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:23:44.043692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543cc7a-5a95-4585-8e3a-096d7c4a3c29 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:44.044224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543cc7a-5a95-4585-8e3a-096d7c4a3c29 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:44.044543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543cc7a-5a95-4585-8e3a-096d7c4a3c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:44.075864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b543cc7a-5a95-4585-8e3a-096d7c4a3c29 None None] MFA Rules not processed for user `fe3669f1e03447f7a5793e454101cde2`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:23:44.077142 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b543cc7a-5a95-4585-8e3a-096d7c4a3c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:44.143623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1283/2553] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:23:44 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:23:44.154904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa071fdb-6e18-4738-bb0e-b6ea8ae2d3b2 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:44.156612 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-aa071fdb-6e18-4738-bb0e-b6ea8ae2d3b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:44.215885 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:44.235775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa071fdb-6e18-4738-bb0e-b6ea8ae2d3b2 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:44.239692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bff939-dfe2-451b-b0b3-9749ba4d9f01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:44.240607 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bff939-dfe2-451b-b0b3-9749ba4d9f01 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:44.241106 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bff939-dfe2-451b-b0b3-9749ba4d9f01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:44.241438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bff939-dfe2-451b-b0b3-9749ba4d9f01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:44.251383 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bff939-dfe2-451b-b0b3-9749ba4d9f01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:44.252927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bff939-dfe2-451b-b0b3-9749ba4d9f01 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:44.262923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1271/2554] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:44 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:23:44.668792 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543cc7a-5a95-4585-8e3a-096d7c4a3c29 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:44.678645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543cc7a-5a95-4585-8e3a-096d7c4a3c29 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:44.682286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698d301-ef7a-444c-aceb-71b698c86806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:44.682900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d301-ef7a-444c-aceb-71b698c86806 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:44.683181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d301-ef7a-444c-aceb-71b698c86806 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:44.683483 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d301-ef7a-444c-aceb-71b698c86806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:44.685358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8698d301-ef7a-444c-aceb-71b698c86806 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:44.735465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:44.745638 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698d301-ef7a-444c-aceb-71b698c86806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:44.747074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698d301-ef7a-444c-aceb-71b698c86806 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:44.757169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1284/2555] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:44 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:23:49.611214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bff939-dfe2-451b-b0b3-9749ba4d9f01 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:49.612611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-72bff939-dfe2-451b-b0b3-9749ba4d9f01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:49.623823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bff939-dfe2-451b-b0b3-9749ba4d9f01 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:49.627432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258a235-3da6-40eb-abe8-e7ac7d2fae9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:49.628036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258a235-3da6-40eb-abe8-e7ac7d2fae9c service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:49.628319 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258a235-3da6-40eb-abe8-e7ac7d2fae9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:49.628593 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258a235-3da6-40eb-abe8-e7ac7d2fae9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:49.630294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2258a235-3da6-40eb-abe8-e7ac7d2fae9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:49.689067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258a235-3da6-40eb-abe8-e7ac7d2fae9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:49.690518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258a235-3da6-40eb-abe8-e7ac7d2fae9c service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:49.700148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1272/2556] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:23:54.806734 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:58.654179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698d301-ef7a-444c-aceb-71b698c86806 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:58.715986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698d301-ef7a-444c-aceb-71b698c86806 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:58.720751 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:58.721427 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:58.721713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:58.721991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] PATH_INFO: `/v3/users/b263ac3c4e3f446c961ba8d715e369da` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:58.723164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b263ac3c4e3f446c961ba8d715e369da)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:58.725118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:58.893075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b263ac3c4e3f446c961ba8d715e369da'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:58.893741 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b263ac3c4e3f446c961ba8d715e369da'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:58.900634 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b263ac3c4e3f446c961ba8d715e369da'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:58.965906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b263ac3c4e3f446c961ba8d715e369da'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:58.966579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b263ac3c4e3f446c961ba8d715e369da'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:58.985383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1285/2557] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:23:58 2022] DELETE /identity/v3/users/b263ac3c4e3f446c961ba8d715e369da => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:23:58.989092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279a1ea-45dd-4c09-a8f0-ce9e6aaa167c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:58.989575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279a1ea-45dd-4c09-a8f0-ce9e6aaa167c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:58.989677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279a1ea-45dd-4c09-a8f0-ce9e6aaa167c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:59.008789 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9279a1ea-45dd-4c09-a8f0-ce9e6aaa167c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:23:59.010051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9279a1ea-45dd-4c09-a8f0-ce9e6aaa167c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:59.068539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1273/2558] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:23:59.078372 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:59.079826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:59.137145 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532030f-8e68-4626-9527-2bff3784bd94 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:59.140797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382f533-0e3b-423a-8a75-5f0caef5c0a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:59.141434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382f533-0e3b-423a-8a75-5f0caef5c0a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:59.141720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382f533-0e3b-423a-8a75-5f0caef5c0a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:59.141998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382f533-0e3b-423a-8a75-5f0caef5c0a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:59.151908 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382f533-0e3b-423a-8a75-5f0caef5c0a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:59.153375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382f533-0e3b-423a-8a75-5f0caef5c0a3 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:59.162813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1286/2559] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:23:59.556760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279a1ea-45dd-4c09-a8f0-ce9e6aaa167c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:59.558292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9279a1ea-45dd-4c09-a8f0-ce9e6aaa167c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:23:59.570728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279a1ea-45dd-4c09-a8f0-ce9e6aaa167c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:59.576050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:59.576700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:59.577005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:59.577287 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] PATH_INFO: `/v3/projects/2b4e63e7c1024cf5a76d41875aad53b6` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:59.578468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b4e63e7c1024cf5a76d41875aad53b6)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:59.580448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:23:59.680905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b4e63e7c1024cf5a76d41875aad53b6', 'request_id': 'req-7d8ce995-8727-4020-a98a-42693f1d5305'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:59.686376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b4e63e7c1024cf5a76d41875aad53b6', 'request_id': 'req-7d8ce995-8727-4020-a98a-42693f1d5305'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:59.736322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b4e63e7c1024cf5a76d41875aad53b6', 'request_id': 'req-7d8ce995-8727-4020-a98a-42693f1d5305'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:59.744756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:59.748468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] The token cache is being invalidate because project 2b4e63e7c1024cf5a76d41875aad53b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:23:59.751258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:59.751258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:23:59.751258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1274/2560] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:23:59 2022] DELETE /identity/v3/projects/2b4e63e7c1024cf5a76d41875aad53b6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:23:59.754918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382f533-0e3b-423a-8a75-5f0caef5c0a3 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:23:59.765530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382f533-0e3b-423a-8a75-5f0caef5c0a3 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:23:59.787672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:23:59.828211 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697579ba-0fb5-4a8b-90f8-dc1598808348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:23:59.828781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697579ba-0fb5-4a8b-90f8-dc1598808348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:23:59.829098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697579ba-0fb5-4a8b-90f8-dc1598808348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:23:59.829384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697579ba-0fb5-4a8b-90f8-dc1598808348 admin admin] PATH_INFO: `/v3/users/fe3669f1e03447f7a5793e454101cde2` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:23:59.830577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697579ba-0fb5-4a8b-90f8-dc1598808348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3669f1e03447f7a5793e454101cde2)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:23:59.832507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697579ba-0fb5-4a8b-90f8-dc1598808348 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:00.041835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-697579ba-0fb5-4a8b-90f8-dc1598808348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3669f1e03447f7a5793e454101cde2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:00.042502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-697579ba-0fb5-4a8b-90f8-dc1598808348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3669f1e03447f7a5793e454101cde2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:00.049378 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-697579ba-0fb5-4a8b-90f8-dc1598808348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3669f1e03447f7a5793e454101cde2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:00.119453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-697579ba-0fb5-4a8b-90f8-dc1598808348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3669f1e03447f7a5793e454101cde2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:00.120074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-697579ba-0fb5-4a8b-90f8-dc1598808348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3669f1e03447f7a5793e454101cde2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:00.138476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1287/2561] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:23:59 2022] DELETE /identity/v3/users/fe3669f1e03447f7a5793e454101cde2 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:00.532857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:00.534434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:00.547538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ce995-8727-4020-a98a-42693f1d5305 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:00.607093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:00.608181 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:00.608689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:00.609107 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] PATH_INFO: `/v3/projects/fd883effc4f942a09ac4aeb6542319fa` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:00.610719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd883effc4f942a09ac4aeb6542319fa)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:00.612799 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:00.726666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd883effc4f942a09ac4aeb6542319fa', 'request_id': 'req-0baf49fa-bec8-4998-bdbf-99a454413a84'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:00.733179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd883effc4f942a09ac4aeb6542319fa', 'request_id': 'req-0baf49fa-bec8-4998-bdbf-99a454413a84'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:00.810896 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd883effc4f942a09ac4aeb6542319fa', 'request_id': 'req-0baf49fa-bec8-4998-bdbf-99a454413a84'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:00.823330 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] The token cache is being invalidate because project fd883effc4f942a09ac4aeb6542319fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:00.829503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:00.829503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-0baf49fa-bec8-4998-bdbf-99a454413a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:00.829503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1275/2562] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:00 2022] DELETE /identity/v3/projects/fd883effc4f942a09ac4aeb6542319fa => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:00.831068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662883a-0595-4382-9ec9-3bfc01e57174 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:00.831515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662883a-0595-4382-9ec9-3bfc01e57174 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:00.831650 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662883a-0595-4382-9ec9-3bfc01e57174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:00.851631 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2662883a-0595-4382-9ec9-3bfc01e57174 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:00.853384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2662883a-0595-4382-9ec9-3bfc01e57174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:00.911653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1288/2563] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:00.919954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945148e-bed2-4218-a425-0f9fe1edd01a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:00.920280 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945148e-bed2-4218-a425-0f9fe1edd01a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:00.920566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945148e-bed2-4218-a425-0f9fe1edd01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:00.941520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8945148e-bed2-4218-a425-0f9fe1edd01a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:00.942775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8945148e-bed2-4218-a425-0f9fe1edd01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:00.950929 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1276/2564] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:00.955156 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662883a-0595-4382-9ec9-3bfc01e57174 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:00.965777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662883a-0595-4382-9ec9-3bfc01e57174 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:00.970774 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf628f3-8f96-4cfa-b59e-5ccdd7818ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:00.971394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf628f3-8f96-4cfa-b59e-5ccdd7818ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:00.971681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf628f3-8f96-4cfa-b59e-5ccdd7818ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:00.971963 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf628f3-8f96-4cfa-b59e-5ccdd7818ae9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:00.972670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf628f3-8f96-4cfa-b59e-5ccdd7818ae9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:24:00.973038 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf628f3-8f96-4cfa-b59e-5ccdd7818ae9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:00.974643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf628f3-8f96-4cfa-b59e-5ccdd7818ae9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:00.987268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1289/2565] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:24:00.991531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945148e-bed2-4218-a425-0f9fe1edd01a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:01.001803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945148e-bed2-4218-a425-0f9fe1edd01a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:01.006611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ec525-0b1a-4d28-b503-959e99b3b6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:01.007232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ec525-0b1a-4d28-b503-959e99b3b6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:01.007519 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ec525-0b1a-4d28-b503-959e99b3b6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:01.007801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ec525-0b1a-4d28-b503-959e99b3b6b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:01.008515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ec525-0b1a-4d28-b503-959e99b3b6b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:01.010560 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ec525-0b1a-4d28-b503-959e99b3b6b3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:01.097600 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1277/2566] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:00 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:01.099076 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf628f3-8f96-4cfa-b59e-5ccdd7818ae9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:01.110190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf628f3-8f96-4cfa-b59e-5ccdd7818ae9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:01.140871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:01.173499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03402e04-471e-4443-93f6-dbceb05022fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:01.174166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03402e04-471e-4443-93f6-dbceb05022fe admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:01.174454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03402e04-471e-4443-93f6-dbceb05022fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:01.174739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03402e04-471e-4443-93f6-dbceb05022fe admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:01.175421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03402e04-471e-4443-93f6-dbceb05022fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:01.177381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03402e04-471e-4443-93f6-dbceb05022fe admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:01.246377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1290/2567] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:01 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:01.250800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ec525-0b1a-4d28-b503-959e99b3b6b3 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:01.261373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ec525-0b1a-4d28-b503-959e99b3b6b3 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:01.266312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b89c1d-02c1-42d6-82f5-1e710244b44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:01.266937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b89c1d-02c1-42d6-82f5-1e710244b44c admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:01.269745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b89c1d-02c1-42d6-82f5-1e710244b44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:01.269745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b89c1d-02c1-42d6-82f5-1e710244b44c admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:01.269745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b89c1d-02c1-42d6-82f5-1e710244b44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:01.270035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b89c1d-02c1-42d6-82f5-1e710244b44c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:01.281518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1278/2568] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:24:01.285830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03402e04-471e-4443-93f6-dbceb05022fe admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:01.296710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03402e04-471e-4443-93f6-dbceb05022fe admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:01.301378 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b00be7-b51e-4881-bd2e-ba24259d2f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:01.301986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b00be7-b51e-4881-bd2e-ba24259d2f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:01.302272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b00be7-b51e-4881-bd2e-ba24259d2f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:01.302549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b00be7-b51e-4881-bd2e-ba24259d2f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:01.303291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b00be7-b51e-4881-bd2e-ba24259d2f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:01.304874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b00be7-b51e-4881-bd2e-ba24259d2f6d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:01.315671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1291/2569] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:01.319820 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b89c1d-02c1-42d6-82f5-1e710244b44c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:01.330834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b89c1d-02c1-42d6-82f5-1e710244b44c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:01.335652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eaac54-fe62-4fa9-8e1a-157763246e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:01.336567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eaac54-fe62-4fa9-8e1a-157763246e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:01.336848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eaac54-fe62-4fa9-8e1a-157763246e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:01.337642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eaac54-fe62-4fa9-8e1a-157763246e13 admin admin] PATH_INFO: `/v3/projects/953962877a484fe0af2bc794d2eb614a/users/81f937c1597e4cfc831c7267afc0f6c3/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:01.352023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eaac54-fe62-4fa9-8e1a-157763246e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=953962877a484fe0af2bc794d2eb614a, user_id=81f937c1597e4cfc831c7267afc0f6c3, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:01.354696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eaac54-fe62-4fa9-8e1a-157763246e13 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:01.400659 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1279/2570] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:01 2022] PUT /identity/v3/projects/953962877a484fe0af2bc794d2eb614a/users/81f937c1597e4cfc831c7267afc0f6c3/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:01.416176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b693d3-8627-4443-b74e-2f874fb7605b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:01.416491 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b693d3-8627-4443-b74e-2f874fb7605b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:01.416776 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b693d3-8627-4443-b74e-2f874fb7605b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:01.448601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d9b693d3-8627-4443-b74e-2f874fb7605b None None] MFA Rules not processed for user `81f937c1597e4cfc831c7267afc0f6c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:01.453688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d9b693d3-8627-4443-b74e-2f874fb7605b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:01.521610 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1292/2571] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:01.526096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eaac54-fe62-4fa9-8e1a-157763246e13 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:01.555425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:01.588261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eaac54-fe62-4fa9-8e1a-157763246e13 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:01.593468 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c649b-a5b0-4422-9f11-6a9675a2dc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:01.594077 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c649b-a5b0-4422-9f11-6a9675a2dc3c admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:01.594357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c649b-a5b0-4422-9f11-6a9675a2dc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:01.594630 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c649b-a5b0-4422-9f11-6a9675a2dc3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:01.595298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c649b-a5b0-4422-9f11-6a9675a2dc3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:01.597232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c649b-a5b0-4422-9f11-6a9675a2dc3c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:01.716676 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1280/2572] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:01 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:01.720812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b693d3-8627-4443-b74e-2f874fb7605b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:01.731579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b693d3-8627-4443-b74e-2f874fb7605b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:01.787279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdee5d7-09dc-43b3-be38-2c1fde41740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:01.787828 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdee5d7-09dc-43b3-be38-2c1fde41740e admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:01.788113 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdee5d7-09dc-43b3-be38-2c1fde41740e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:01.788393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdee5d7-09dc-43b3-be38-2c1fde41740e admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:01.789160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdee5d7-09dc-43b3-be38-2c1fde41740e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:01.793701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdee5d7-09dc-43b3-be38-2c1fde41740e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:01.873721 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1293/2573] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:01 2022] POST /identity/v3/users => generated 299 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:01.877765 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c649b-a5b0-4422-9f11-6a9675a2dc3c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:01.888571 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c649b-a5b0-4422-9f11-6a9675a2dc3c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:01.893247 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97456b-e7df-4a70-9633-894722866918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:01.893857 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97456b-e7df-4a70-9633-894722866918 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:01.894148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97456b-e7df-4a70-9633-894722866918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:01.894418 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97456b-e7df-4a70-9633-894722866918 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:01.895114 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97456b-e7df-4a70-9633-894722866918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:01.896707 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97456b-e7df-4a70-9633-894722866918 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:01.907064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1281/2574] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:01.910975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdee5d7-09dc-43b3-be38-2c1fde41740e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:01.920870 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdee5d7-09dc-43b3-be38-2c1fde41740e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:01.925401 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f69d9e-8e44-4de8-9b20-52a4a17aab22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:01.926055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f69d9e-8e44-4de8-9b20-52a4a17aab22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:01.926358 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f69d9e-8e44-4de8-9b20-52a4a17aab22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:01.926637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f69d9e-8e44-4de8-9b20-52a4a17aab22 admin admin] PATH_INFO: `/v3/projects/4c851a8c036144f890be99b679184eba/users/321c8d8dedcb4c9897d18291f00a594c/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:01.940954 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f69d9e-8e44-4de8-9b20-52a4a17aab22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c851a8c036144f890be99b679184eba, user_id=321c8d8dedcb4c9897d18291f00a594c, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:01.943612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f69d9e-8e44-4de8-9b20-52a4a17aab22 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:02.005046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1294/2575] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:01 2022] PUT /identity/v3/projects/4c851a8c036144f890be99b679184eba/users/321c8d8dedcb4c9897d18291f00a594c/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:02.009701 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97456b-e7df-4a70-9633-894722866918 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:02.020026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97456b-e7df-4a70-9633-894722866918 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:02.075855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e61afb-3437-4a00-9484-7b74a56d6e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:02.076397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e61afb-3437-4a00-9484-7b74a56d6e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:02.076686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e61afb-3437-4a00-9484-7b74a56d6e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:02.076981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e61afb-3437-4a00-9484-7b74a56d6e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:02.077699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e61afb-3437-4a00-9484-7b74a56d6e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:02.079294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e61afb-3437-4a00-9484-7b74a56d6e89 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:02.089197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1282/2576] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:02.093303 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f69d9e-8e44-4de8-9b20-52a4a17aab22 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:02.098765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:02.104252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f69d9e-8e44-4de8-9b20-52a4a17aab22 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:02.110436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20b351-d9e6-4474-b42e-87403107f058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:02.110436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20b351-d9e6-4474-b42e-87403107f058 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:02.110436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20b351-d9e6-4474-b42e-87403107f058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:02.110436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20b351-d9e6-4474-b42e-87403107f058 admin admin] PATH_INFO: `/v3/projects/4c851a8c036144f890be99b679184eba/users/321c8d8dedcb4c9897d18291f00a594c/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:02.112221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20b351-d9e6-4474-b42e-87403107f058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c851a8c036144f890be99b679184eba, user_id=321c8d8dedcb4c9897d18291f00a594c, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:02.114980 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20b351-d9e6-4474-b42e-87403107f058 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:02.126109 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1295/2577] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:02 2022] PUT /identity/v3/projects/4c851a8c036144f890be99b679184eba/users/321c8d8dedcb4c9897d18291f00a594c/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:02.137566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ef5de-843b-4def-adaf-4c2e6c60d489 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:02.137858 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ef5de-843b-4def-adaf-4c2e6c60d489 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:02.138137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ef5de-843b-4def-adaf-4c2e6c60d489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:02.166169 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-4a5ef5de-843b-4def-adaf-4c2e6c60d489 None None] MFA Rules not processed for user `321c8d8dedcb4c9897d18291f00a594c`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:02.167423 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4a5ef5de-843b-4def-adaf-4c2e6c60d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:02.238147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1283/2578] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:02.249447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20b351-d9e6-4474-b42e-87403107f058 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:02.251228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ad20b351-d9e6-4474-b42e-87403107f058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:02.307824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20b351-d9e6-4474-b42e-87403107f058 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:02.311554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7769a-2a40-456c-9521-4e092db3d996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:02.312174 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7769a-2a40-456c-9521-4e092db3d996 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:02.312457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7769a-2a40-456c-9521-4e092db3d996 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:02.312733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7769a-2a40-456c-9521-4e092db3d996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:02.321919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7769a-2a40-456c-9521-4e092db3d996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:02.323342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7769a-2a40-456c-9521-4e092db3d996 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:02.332806 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1296/2579] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:03.420399 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ef5de-843b-4def-adaf-4c2e6c60d489 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:03.446034 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:03.483797 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ef5de-843b-4def-adaf-4c2e6c60d489 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:03.488666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:03.490497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:03.490497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:03.490497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] PATH_INFO: `/v3/users/81f937c1597e4cfc831c7267afc0f6c3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:03.491060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f937c1597e4cfc831c7267afc0f6c3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:03.493033 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:03.676617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f937c1597e4cfc831c7267afc0f6c3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:03.683663 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f937c1597e4cfc831c7267afc0f6c3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:03.736836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f937c1597e4cfc831c7267afc0f6c3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:03.737489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f937c1597e4cfc831c7267afc0f6c3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:03.749619 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f937c1597e4cfc831c7267afc0f6c3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:03.755474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1284/2580] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:03 2022] DELETE /identity/v3/users/81f937c1597e4cfc831c7267afc0f6c3 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:03.759956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0280be-0a3c-4385-aafe-a685bff2de74 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:03.760540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0280be-0a3c-4385-aafe-a685bff2de74 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:03.760652 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0280be-0a3c-4385-aafe-a685bff2de74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:03.779760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8d0280be-0a3c-4385-aafe-a685bff2de74 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:03.781001 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8d0280be-0a3c-4385-aafe-a685bff2de74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:03.841703 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1297/2581] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:03.851867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:03.854040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:03.912393 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd6b6a-06dc-4080-a73a-a47fec42a20f admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:03.916132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b3f6a-a3b5-4d50-ad09-f806d874c622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:03.916751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3f6a-a3b5-4d50-ad09-f806d874c622 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:03.917067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3f6a-a3b5-4d50-ad09-f806d874c622 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:03.917347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3f6a-a3b5-4d50-ad09-f806d874c622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:03.926312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b3f6a-a3b5-4d50-ad09-f806d874c622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:03.927800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b3f6a-a3b5-4d50-ad09-f806d874c622 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:03.937272 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1285/2582] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:04.346378 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0280be-0a3c-4385-aafe-a685bff2de74 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:04.346792 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8d0280be-0a3c-4385-aafe-a685bff2de74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:04.359259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0280be-0a3c-4385-aafe-a685bff2de74 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:04.364044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d53739-f78b-4e50-aea8-14e48e036442 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:04.364263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:04.364409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d53739-f78b-4e50-aea8-14e48e036442 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:04.364604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d53739-f78b-4e50-aea8-14e48e036442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:04.365285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:04.365403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:04.365883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] PATH_INFO: `/v3/projects/953962877a484fe0af2bc794d2eb614a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:04.367139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=953962877a484fe0af2bc794d2eb614a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:04.369746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:04.387378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-89d53739-f78b-4e50-aea8-14e48e036442 None None] MFA Rules not processed for user `30f0002d2a5a4950b2f1aa771f33bbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:04.391920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-89d53739-f78b-4e50-aea8-14e48e036442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:04.429708 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:04.445368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1286/2583] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:04.488129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '953962877a484fe0af2bc794d2eb614a', 'request_id': 'req-fac9c575-e466-4c42-a530-cd0a29dca100'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:04.494357 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '953962877a484fe0af2bc794d2eb614a', 'request_id': 'req-fac9c575-e466-4c42-a530-cd0a29dca100'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:04.500545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '953962877a484fe0af2bc794d2eb614a', 'request_id': 'req-fac9c575-e466-4c42-a530-cd0a29dca100'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:04.582662 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] The token cache is being invalidate because project 953962877a484fe0af2bc794d2eb614a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:04.583208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:04.586567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:04.586567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1298/2584] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:04 2022] DELETE /identity/v3/projects/953962877a484fe0af2bc794d2eb614a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:04.589715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d53739-f78b-4e50-aea8-14e48e036442 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:04.600942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d53739-f78b-4e50-aea8-14e48e036442 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:04.662836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ac491-071d-4392-8799-4a2276ea0f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:04.663493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ac491-071d-4392-8799-4a2276ea0f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:04.663776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ac491-071d-4392-8799-4a2276ea0f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:04.664053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ac491-071d-4392-8799-4a2276ea0f5a admin admin] PATH_INFO: `/v3/users/321c8d8dedcb4c9897d18291f00a594c` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:04.665775 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ac491-071d-4392-8799-4a2276ea0f5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=321c8d8dedcb4c9897d18291f00a594c)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:04.667710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ac491-071d-4392-8799-4a2276ea0f5a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:04.854664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a53ac491-071d-4392-8799-4a2276ea0f5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '321c8d8dedcb4c9897d18291f00a594c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:04.862651 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a53ac491-071d-4392-8799-4a2276ea0f5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '321c8d8dedcb4c9897d18291f00a594c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:04.908611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a53ac491-071d-4392-8799-4a2276ea0f5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '321c8d8dedcb4c9897d18291f00a594c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:04.909307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a53ac491-071d-4392-8799-4a2276ea0f5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '321c8d8dedcb4c9897d18291f00a594c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:04.922444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-a53ac491-071d-4392-8799-4a2276ea0f5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '321c8d8dedcb4c9897d18291f00a594c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:04.928688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1287/2585] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:04 2022] DELETE /identity/v3/users/321c8d8dedcb4c9897d18291f00a594c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:05.362906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:05.365442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:05.382548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9c575-e466-4c42-a530-cd0a29dca100 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:05.395166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:05.454035 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:05.454035 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:05.454035 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:05.454035 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] PATH_INFO: `/v3/projects/4c851a8c036144f890be99b679184eba` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:05.454432 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c851a8c036144f890be99b679184eba)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:05.455157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:05.550646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c851a8c036144f890be99b679184eba', 'request_id': 'req-d8cbf521-6df8-43d9-b2d0-dd66977f580e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:05.556755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c851a8c036144f890be99b679184eba', 'request_id': 'req-d8cbf521-6df8-43d9-b2d0-dd66977f580e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:05.563088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c851a8c036144f890be99b679184eba', 'request_id': 'req-d8cbf521-6df8-43d9-b2d0-dd66977f580e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:05.633994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] The token cache is being invalidate because project 4c851a8c036144f890be99b679184eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:05.634288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:05.634909 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d8cbf521-6df8-43d9-b2d0-dd66977f580e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:05.636509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1299/2586] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:05 2022] DELETE /identity/v3/projects/4c851a8c036144f890be99b679184eba => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:05.642099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dea0d-42a8-4979-af70-c17ab5cc948e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:05.642588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dea0d-42a8-4979-af70-c17ab5cc948e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:05.642706 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dea0d-42a8-4979-af70-c17ab5cc948e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:05.662023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-683dea0d-42a8-4979-af70-c17ab5cc948e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:05.663267 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-683dea0d-42a8-4979-af70-c17ab5cc948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:05.725470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1288/2587] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:05.733917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6b231-6819-4e85-8165-d4c97a3dab92 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:05.734205 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6b231-6819-4e85-8165-d4c97a3dab92 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:05.734480 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6b231-6819-4e85-8165-d4c97a3dab92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:05.754917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-dad6b231-6819-4e85-8165-d4c97a3dab92 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:05.756196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dad6b231-6819-4e85-8165-d4c97a3dab92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:05.766676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1300/2588] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:05.768832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dea0d-42a8-4979-af70-c17ab5cc948e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:05.779822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dea0d-42a8-4979-af70-c17ab5cc948e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:05.784588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c18db-f918-4b1e-8ccd-711a5cdf9715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:05.785246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c18db-f918-4b1e-8ccd-711a5cdf9715 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:05.785531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c18db-f918-4b1e-8ccd-711a5cdf9715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:05.785808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c18db-f918-4b1e-8ccd-711a5cdf9715 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:05.786514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c18db-f918-4b1e-8ccd-711a5cdf9715 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:24:05.786821 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c18db-f918-4b1e-8ccd-711a5cdf9715 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:05.788430 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c18db-f918-4b1e-8ccd-711a5cdf9715 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:05.791812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:05.804344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1289/2589] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:24:05.805514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6b231-6819-4e85-8165-d4c97a3dab92 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:05.815062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6b231-6819-4e85-8165-d4c97a3dab92 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:05.820152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cd0ad-29f9-4fa2-a483-feb3447a91a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:05.820762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cd0ad-29f9-4fa2-a483-feb3447a91a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:05.821085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cd0ad-29f9-4fa2-a483-feb3447a91a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:05.821558 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cd0ad-29f9-4fa2-a483-feb3447a91a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:05.822222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cd0ad-29f9-4fa2-a483-feb3447a91a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:05.824334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cd0ad-29f9-4fa2-a483-feb3447a91a1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:05.917414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1301/2590] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:05 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:05.921683 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c18db-f918-4b1e-8ccd-711a5cdf9715 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:05.931923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c18db-f918-4b1e-8ccd-711a5cdf9715 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:05.989606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10191599-17a8-4da3-9d7d-e5cbbab5f8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:05.990233 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10191599-17a8-4da3-9d7d-e5cbbab5f8cb admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:05.990518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10191599-17a8-4da3-9d7d-e5cbbab5f8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:05.990796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10191599-17a8-4da3-9d7d-e5cbbab5f8cb admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:05.991476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10191599-17a8-4da3-9d7d-e5cbbab5f8cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:05.993532 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10191599-17a8-4da3-9d7d-e5cbbab5f8cb admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:06.111473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1290/2591] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:05 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:06.113406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cd0ad-29f9-4fa2-a483-feb3447a91a1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:06.123973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cd0ad-29f9-4fa2-a483-feb3447a91a1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:06.128864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fa871-e813-4ed6-ae06-6a709bfb4b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:06.129514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fa871-e813-4ed6-ae06-6a709bfb4b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:06.129800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fa871-e813-4ed6-ae06-6a709bfb4b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:06.130078 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fa871-e813-4ed6-ae06-6a709bfb4b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:06.130781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2fa871-e813-4ed6-ae06-6a709bfb4b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:06.132445 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2fa871-e813-4ed6-ae06-6a709bfb4b02 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:06.143524 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1302/2592] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:06.147632 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10191599-17a8-4da3-9d7d-e5cbbab5f8cb admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:06.158543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10191599-17a8-4da3-9d7d-e5cbbab5f8cb admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:06.163637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd893b-fa9a-4960-aa6a-457f43e0adcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:06.166879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd893b-fa9a-4960-aa6a-457f43e0adcd admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:06.166879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd893b-fa9a-4960-aa6a-457f43e0adcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:06.166879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd893b-fa9a-4960-aa6a-457f43e0adcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:06.166879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd893b-fa9a-4960-aa6a-457f43e0adcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:06.167337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd893b-fa9a-4960-aa6a-457f43e0adcd admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:06.178192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1291/2593] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:06.182397 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fa871-e813-4ed6-ae06-6a709bfb4b02 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:06.192709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fa871-e813-4ed6-ae06-6a709bfb4b02 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:06.198055 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815b5b3-6340-435f-aecc-0a8c49c772f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:06.198717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815b5b3-6340-435f-aecc-0a8c49c772f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:06.199004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815b5b3-6340-435f-aecc-0a8c49c772f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:06.199286 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815b5b3-6340-435f-aecc-0a8c49c772f0 admin admin] PATH_INFO: `/v3/projects/11228e5b5bf34360b2a65a638214b95c/users/d02345b7354546de8bf06a9a38362386/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:06.214383 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815b5b3-6340-435f-aecc-0a8c49c772f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11228e5b5bf34360b2a65a638214b95c, user_id=d02345b7354546de8bf06a9a38362386, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:06.217068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815b5b3-6340-435f-aecc-0a8c49c772f0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:06.315103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1303/2594] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:06 2022] PUT /identity/v3/projects/11228e5b5bf34360b2a65a638214b95c/users/d02345b7354546de8bf06a9a38362386/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:06.326141 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2820499-9ecd-416b-bf70-607caec16818 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:06.326433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2820499-9ecd-416b-bf70-607caec16818 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:06.326741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2820499-9ecd-416b-bf70-607caec16818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:06.356182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e2820499-9ecd-416b-bf70-607caec16818 None None] MFA Rules not processed for user `d02345b7354546de8bf06a9a38362386`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:06.357465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e2820499-9ecd-416b-bf70-607caec16818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:06.424347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1292/2595] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:06.436093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815b5b3-6340-435f-aecc-0a8c49c772f0 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:06.438065 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4815b5b3-6340-435f-aecc-0a8c49c772f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:06.485495 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:06.496832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815b5b3-6340-435f-aecc-0a8c49c772f0 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:06.502498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4d140-a763-4ed7-a155-21003207be9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:06.502498 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4d140-a763-4ed7-a155-21003207be9c service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:06.502759 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4d140-a763-4ed7-a155-21003207be9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:06.503051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4d140-a763-4ed7-a155-21003207be9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:06.512800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4d140-a763-4ed7-a155-21003207be9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:06.513339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4d140-a763-4ed7-a155-21003207be9c service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:06.522760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1304/2596] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:06.544166 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2820499-9ecd-416b-bf70-607caec16818 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:06.641506 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2820499-9ecd-416b-bf70-607caec16818 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:06.646241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:06.646788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:06.647062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:06.647337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] PATH_INFO: `/v3/users/d02345b7354546de8bf06a9a38362386` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:06.648526 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02345b7354546de8bf06a9a38362386)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:06.650472 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:06.655791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:06.801776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02345b7354546de8bf06a9a38362386'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:06.808525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02345b7354546de8bf06a9a38362386'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:06.908528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02345b7354546de8bf06a9a38362386'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:06.909237 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02345b7354546de8bf06a9a38362386'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:06.921481 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02345b7354546de8bf06a9a38362386'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:06.927352 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1293/2597] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:06 2022] DELETE /identity/v3/users/d02345b7354546de8bf06a9a38362386 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:06.932044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dffcf2-bc33-4a79-a0a1-ad42c32ded83 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:06.932334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dffcf2-bc33-4a79-a0a1-ad42c32ded83 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:06.932633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dffcf2-bc33-4a79-a0a1-ad42c32ded83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:06.954902 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f0dffcf2-bc33-4a79-a0a1-ad42c32ded83 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:06.956035 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f0dffcf2-bc33-4a79-a0a1-ad42c32ded83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:07.016114 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1305/2598] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:07.026594 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:07.028154 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:07.091202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2499f-cedc-44ee-aad1-4c2fe1136d9c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:07.094901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c655a-312b-423a-82b1-6bf6032b4ffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:07.095652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c655a-312b-423a-82b1-6bf6032b4ffb service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:07.095947 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c655a-312b-423a-82b1-6bf6032b4ffb service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:07.096260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c655a-312b-423a-82b1-6bf6032b4ffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:07.106175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c655a-312b-423a-82b1-6bf6032b4ffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:07.107617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c655a-312b-423a-82b1-6bf6032b4ffb service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:07.117673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1294/2599] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:07.555230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dffcf2-bc33-4a79-a0a1-ad42c32ded83 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:07.556754 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f0dffcf2-bc33-4a79-a0a1-ad42c32ded83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:07.569975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dffcf2-bc33-4a79-a0a1-ad42c32ded83 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:07.575463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:07.576149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:07.576586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:07.576871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] PATH_INFO: `/v3/projects/11228e5b5bf34360b2a65a638214b95c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:07.578102 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=11228e5b5bf34360b2a65a638214b95c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:07.580018 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:07.688860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11228e5b5bf34360b2a65a638214b95c', 'request_id': 'req-50b6ef31-c140-4743-980f-e122e680de0b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:07.694497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11228e5b5bf34360b2a65a638214b95c', 'request_id': 'req-50b6ef31-c140-4743-980f-e122e680de0b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:07.700825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11228e5b5bf34360b2a65a638214b95c', 'request_id': 'req-50b6ef31-c140-4743-980f-e122e680de0b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:07.777092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] The token cache is being invalidate because project 11228e5b5bf34360b2a65a638214b95c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:07.777394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:07.778036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-50b6ef31-c140-4743-980f-e122e680de0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:07.779574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1306/2600] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:07 2022] DELETE /identity/v3/projects/11228e5b5bf34360b2a65a638214b95c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:07.785328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd39ea-955e-4b9c-98e1-de074c3077e2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:07.785842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd39ea-955e-4b9c-98e1-de074c3077e2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:07.785945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd39ea-955e-4b9c-98e1-de074c3077e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:07.806204 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-35dd39ea-955e-4b9c-98e1-de074c3077e2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:07.807958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-35dd39ea-955e-4b9c-98e1-de074c3077e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:07.867932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1295/2601] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:07.877313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46570300-a6aa-4811-96fc-a3b3d12bdd79 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:07.877608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46570300-a6aa-4811-96fc-a3b3d12bdd79 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:07.877891 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46570300-a6aa-4811-96fc-a3b3d12bdd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:07.897196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-46570300-a6aa-4811-96fc-a3b3d12bdd79 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:07.898415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-46570300-a6aa-4811-96fc-a3b3d12bdd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:07.907026 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1307/2602] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:07.911374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd39ea-955e-4b9c-98e1-de074c3077e2 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:07.922620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd39ea-955e-4b9c-98e1-de074c3077e2 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:07.927294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41bc12-edb1-4c44-a349-0fec1e4acfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:07.927914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41bc12-edb1-4c44-a349-0fec1e4acfc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:07.928200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41bc12-edb1-4c44-a349-0fec1e4acfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:07.928482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41bc12-edb1-4c44-a349-0fec1e4acfc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:07.929220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41bc12-edb1-4c44-a349-0fec1e4acfc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:24:07.929529 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41bc12-edb1-4c44-a349-0fec1e4acfc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:07.931125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41bc12-edb1-4c44-a349-0fec1e4acfc2 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:07.943842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1296/2603] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:24:07.948157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46570300-a6aa-4811-96fc-a3b3d12bdd79 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:07.957537 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46570300-a6aa-4811-96fc-a3b3d12bdd79 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:07.962551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88876ee1-61bd-4dd3-adb4-8e0709b118d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:07.963277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88876ee1-61bd-4dd3-adb4-8e0709b118d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:07.964068 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88876ee1-61bd-4dd3-adb4-8e0709b118d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:07.964351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88876ee1-61bd-4dd3-adb4-8e0709b118d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:07.965072 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88876ee1-61bd-4dd3-adb4-8e0709b118d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:07.966975 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88876ee1-61bd-4dd3-adb4-8e0709b118d3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:08.058052 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1308/2604] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:07 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:24:08.062102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41bc12-edb1-4c44-a349-0fec1e4acfc2 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:08.072454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41bc12-edb1-4c44-a349-0fec1e4acfc2 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:08.081302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:08.255138 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eff580-f147-4178-a8d2-6eabca3b9f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:08.255783 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eff580-f147-4178-a8d2-6eabca3b9f7a admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:08.256588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eff580-f147-4178-a8d2-6eabca3b9f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:08.256882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eff580-f147-4178-a8d2-6eabca3b9f7a admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:08.257606 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eff580-f147-4178-a8d2-6eabca3b9f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:08.259619 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eff580-f147-4178-a8d2-6eabca3b9f7a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:08.326982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1297/2605] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:08 2022] POST /identity/v3/users => generated 286 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:08.330991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88876ee1-61bd-4dd3-adb4-8e0709b118d3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:08.342921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88876ee1-61bd-4dd3-adb4-8e0709b118d3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:08.347739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71b58e-e965-423e-a2bc-17d8fb98d9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:08.348360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71b58e-e965-423e-a2bc-17d8fb98d9fd admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:08.348643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71b58e-e965-423e-a2bc-17d8fb98d9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:08.348922 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71b58e-e965-423e-a2bc-17d8fb98d9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:08.349657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71b58e-e965-423e-a2bc-17d8fb98d9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:08.351252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71b58e-e965-423e-a2bc-17d8fb98d9fd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:08.361733 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1309/2606] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:08.365779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eff580-f147-4178-a8d2-6eabca3b9f7a admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:08.375901 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eff580-f147-4178-a8d2-6eabca3b9f7a admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:08.380927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3f97b-e5b9-46d4-b74d-b791f262ede6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:08.381569 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3f97b-e5b9-46d4-b74d-b791f262ede6 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:08.381850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3f97b-e5b9-46d4-b74d-b791f262ede6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:08.382205 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3f97b-e5b9-46d4-b74d-b791f262ede6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:08.382954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3f97b-e5b9-46d4-b74d-b791f262ede6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:08.384538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3f97b-e5b9-46d4-b74d-b791f262ede6 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:08.395430 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1298/2607] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:24:08.401623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71b58e-e965-423e-a2bc-17d8fb98d9fd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:08.417966 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71b58e-e965-423e-a2bc-17d8fb98d9fd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:08.424048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295fd1c-55e2-4bf8-97ad-c9bc1b2f3ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:08.424048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295fd1c-55e2-4bf8-97ad-c9bc1b2f3ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:08.424279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295fd1c-55e2-4bf8-97ad-c9bc1b2f3ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:08.424539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295fd1c-55e2-4bf8-97ad-c9bc1b2f3ae3 admin admin] PATH_INFO: `/v3/projects/3a042522bc0e4aa99b12a7e56af576de/users/4b1e2524f81b4e3689521e3b22ae633e/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:08.439941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295fd1c-55e2-4bf8-97ad-c9bc1b2f3ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a042522bc0e4aa99b12a7e56af576de, user_id=4b1e2524f81b4e3689521e3b22ae633e, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:08.442646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295fd1c-55e2-4bf8-97ad-c9bc1b2f3ae3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:08.528039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1310/2608] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:08 2022] PUT /identity/v3/projects/3a042522bc0e4aa99b12a7e56af576de/users/4b1e2524f81b4e3689521e3b22ae633e/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:08.538324 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7093e-67c9-4035-8d94-3cef24c4aa13 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:08.538641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7093e-67c9-4035-8d94-3cef24c4aa13 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:08.538919 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7093e-67c9-4035-8d94-3cef24c4aa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:08.570158 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5cf7093e-67c9-4035-8d94-3cef24c4aa13 None None] MFA Rules not processed for user `4b1e2524f81b4e3689521e3b22ae633e`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:08.571387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5cf7093e-67c9-4035-8d94-3cef24c4aa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:08.637735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1299/2609] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:08.648870 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295fd1c-55e2-4bf8-97ad-c9bc1b2f3ae3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:08.650681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a295fd1c-55e2-4bf8-97ad-c9bc1b2f3ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:08.657836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:08.715265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295fd1c-55e2-4bf8-97ad-c9bc1b2f3ae3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:08.718998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bf461-5f7d-476a-bc95-9a0609eeb439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:08.719618 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bf461-5f7d-476a-bc95-9a0609eeb439 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:08.719934 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bf461-5f7d-476a-bc95-9a0609eeb439 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:08.720234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bf461-5f7d-476a-bc95-9a0609eeb439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:08.728595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bf461-5f7d-476a-bc95-9a0609eeb439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:08.730088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bf461-5f7d-476a-bc95-9a0609eeb439 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:08.738926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1311/2610] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:08.765780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7093e-67c9-4035-8d94-3cef24c4aa13 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:08.861635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7093e-67c9-4035-8d94-3cef24c4aa13 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:08.861635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:08.861635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:08.861635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:08.862490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] PATH_INFO: `/v3/users/4b1e2524f81b4e3689521e3b22ae633e` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:08.862490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b1e2524f81b4e3689521e3b22ae633e)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:08.862490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:08.886066 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:09.037470 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b1e2524f81b4e3689521e3b22ae633e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:09.048148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b1e2524f81b4e3689521e3b22ae633e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:09.120543 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b1e2524f81b4e3689521e3b22ae633e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:09.121289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b1e2524f81b4e3689521e3b22ae633e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:09.134303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b1e2524f81b4e3689521e3b22ae633e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:09.140385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1300/2611] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:08 2022] DELETE /identity/v3/users/4b1e2524f81b4e3689521e3b22ae633e => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:09.145426 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab834ec1-2a37-4816-8729-dc4a09d14eae None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:09.149517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab834ec1-2a37-4816-8729-dc4a09d14eae None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:09.149517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab834ec1-2a37-4816-8729-dc4a09d14eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:09.165449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ab834ec1-2a37-4816-8729-dc4a09d14eae None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:09.167265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ab834ec1-2a37-4816-8729-dc4a09d14eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:09.235440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1312/2612] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:09.245953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:09.247429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:09.311046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93aa18-7630-40f3-b479-3a2fce3ced6f admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:09.314940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d842e-ffb1-40f8-a44e-b095149f74c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:09.315568 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d842e-ffb1-40f8-a44e-b095149f74c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:09.315847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d842e-ffb1-40f8-a44e-b095149f74c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:09.316119 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d842e-ffb1-40f8-a44e-b095149f74c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:09.326027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d842e-ffb1-40f8-a44e-b095149f74c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:09.327462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d842e-ffb1-40f8-a44e-b095149f74c5 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:09.338874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1301/2613] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:10.634375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab834ec1-2a37-4816-8729-dc4a09d14eae None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:10.634375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ab834ec1-2a37-4816-8729-dc4a09d14eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:10.647639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab834ec1-2a37-4816-8729-dc4a09d14eae None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:10.655301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:10.655301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:10.655301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:10.655301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] PATH_INFO: `/v3/projects/3a042522bc0e4aa99b12a7e56af576de` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:10.667047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a042522bc0e4aa99b12a7e56af576de)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:10.668882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:10.881118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d842e-ffb1-40f8-a44e-b095149f74c5 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:10.883483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-363d842e-ffb1-40f8-a44e-b095149f74c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:10.891549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a042522bc0e4aa99b12a7e56af576de', 'request_id': 'req-806f0626-5e3b-4c22-ac3a-3978e05b6898'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:10.899174 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a042522bc0e4aa99b12a7e56af576de', 'request_id': 'req-806f0626-5e3b-4c22-ac3a-3978e05b6898'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:10.907691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a042522bc0e4aa99b12a7e56af576de', 'request_id': 'req-806f0626-5e3b-4c22-ac3a-3978e05b6898'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:10.991248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] The token cache is being invalidate because project 3a042522bc0e4aa99b12a7e56af576de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:10.991638 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:10.992617 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-806f0626-5e3b-4c22-ac3a-3978e05b6898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:10.996216 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1313/2614] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:10 2022] DELETE /identity/v3/projects/3a042522bc0e4aa99b12a7e56af576de => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:11.000482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc1aa1-d582-4455-b606-ec2e769be94f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:11.006284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc1aa1-d582-4455-b606-ec2e769be94f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:11.006794 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc1aa1-d582-4455-b606-ec2e769be94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:11.045755 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-90dc1aa1-d582-4455-b606-ec2e769be94f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:11.047306 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-90dc1aa1-d582-4455-b606-ec2e769be94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:11.060380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d842e-ffb1-40f8-a44e-b095149f74c5 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:11.122422 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:11.166669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:11.216955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1314/2615] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:11.224502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b9a6a-35c9-448e-adad-edba85270892 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:11.224873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b9a6a-35c9-448e-adad-edba85270892 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:11.225083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b9a6a-35c9-448e-adad-edba85270892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:11.241499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7bf05-5afa-48b9-8a58-7d21eb05a3d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:11.244038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7bf05-5afa-48b9-8a58-7d21eb05a3d5 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:11.245034 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7bf05-5afa-48b9-8a58-7d21eb05a3d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:11.247682 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7bf05-5afa-48b9-8a58-7d21eb05a3d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:11.258000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-67f7bf05-5afa-48b9-8a58-7d21eb05a3d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:11.277848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7bf05-5afa-48b9-8a58-7d21eb05a3d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:11.292253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7bf05-5afa-48b9-8a58-7d21eb05a3d5 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:11.292373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b02b9a6a-35c9-448e-adad-edba85270892 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:11.292373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b02b9a6a-35c9-448e-adad-edba85270892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:11.302035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1302/2616] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:24:11.309053 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1315/2617] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:11.313171 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7bf05-5afa-48b9-8a58-7d21eb05a3d5 service nova] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:11.335752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7bf05-5afa-48b9-8a58-7d21eb05a3d5 service nova] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:11.347002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5600b-7865-4569-b6ff-860d608da391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:11.349258 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5600b-7865-4569-b6ff-860d608da391 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:11.349738 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5600b-7865-4569-b6ff-860d608da391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:11.350133 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5600b-7865-4569-b6ff-860d608da391 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:11.351377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5600b-7865-4569-b6ff-860d608da391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:24:11.351780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5600b-7865-4569-b6ff-860d608da391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:11.354671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5600b-7865-4569-b6ff-860d608da391 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:11.371093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1303/2618] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:24:11.375517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b9a6a-35c9-448e-adad-edba85270892 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:11.390322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b9a6a-35c9-448e-adad-edba85270892 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:11.395207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b14cf6-0b8b-4960-83d1-12ea1d615d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:11.395829 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b14cf6-0b8b-4960-83d1-12ea1d615d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:11.396112 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b14cf6-0b8b-4960-83d1-12ea1d615d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:11.397118 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b14cf6-0b8b-4960-83d1-12ea1d615d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:11.397800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b14cf6-0b8b-4960-83d1-12ea1d615d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:11.399708 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b14cf6-0b8b-4960-83d1-12ea1d615d67 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:11.561465 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1316/2619] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:11 2022] POST /identity/v3/projects => generated 375 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:11.573733 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5600b-7865-4569-b6ff-860d608da391 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:11.584071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5600b-7865-4569-b6ff-860d608da391 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:11.649785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81e014-73c1-4e9b-b702-d24bb6c71b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:11.650355 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81e014-73c1-4e9b-b702-d24bb6c71b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:11.650638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81e014-73c1-4e9b-b702-d24bb6c71b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:11.651592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81e014-73c1-4e9b-b702-d24bb6c71b06 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:11.652273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81e014-73c1-4e9b-b702-d24bb6c71b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:11.654388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81e014-73c1-4e9b-b702-d24bb6c71b06 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:11.731825 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1304/2620] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:11 2022] POST /identity/v3/users => generated 288 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:11.735911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b14cf6-0b8b-4960-83d1-12ea1d615d67 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:11.747825 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b14cf6-0b8b-4960-83d1-12ea1d615d67 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:11.752595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d90e9-6bb9-452c-8772-49efcce4272e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:11.753255 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d90e9-6bb9-452c-8772-49efcce4272e admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:11.753574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d90e9-6bb9-452c-8772-49efcce4272e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:11.753969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d90e9-6bb9-452c-8772-49efcce4272e admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:11.754684 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d90e9-6bb9-452c-8772-49efcce4272e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:11.756486 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d90e9-6bb9-452c-8772-49efcce4272e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:11.770098 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1317/2621] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:11.771312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81e014-73c1-4e9b-b702-d24bb6c71b06 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:11.782298 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81e014-73c1-4e9b-b702-d24bb6c71b06 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:11.786993 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa1bbd-817d-41ae-a251-b9508d7ea898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:11.787627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1bbd-817d-41ae-a251-b9508d7ea898 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:11.787915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1bbd-817d-41ae-a251-b9508d7ea898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:11.788188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1bbd-817d-41ae-a251-b9508d7ea898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:11.788896 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa1bbd-817d-41ae-a251-b9508d7ea898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:11.790506 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa1bbd-817d-41ae-a251-b9508d7ea898 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:11.801634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1305/2622] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:11.805734 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d90e9-6bb9-452c-8772-49efcce4272e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:11.817698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d90e9-6bb9-452c-8772-49efcce4272e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:11.822656 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7feb927-c3c1-4051-88ad-da0aca5a4732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:11.823898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7feb927-c3c1-4051-88ad-da0aca5a4732 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:11.824183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7feb927-c3c1-4051-88ad-da0aca5a4732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:11.824463 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7feb927-c3c1-4051-88ad-da0aca5a4732 admin admin] PATH_INFO: `/v3/projects/fa13cc57b22c438b842a59ff4e207f10/users/f30ea147cc194c2f84add802a6ff624f/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:11.841453 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7feb927-c3c1-4051-88ad-da0aca5a4732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa13cc57b22c438b842a59ff4e207f10, user_id=f30ea147cc194c2f84add802a6ff624f, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:11.844405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7feb927-c3c1-4051-88ad-da0aca5a4732 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:11.926561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1318/2623] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:11 2022] PUT /identity/v3/projects/fa13cc57b22c438b842a59ff4e207f10/users/f30ea147cc194c2f84add802a6ff624f/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:11.938504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca14a5-6a63-4a9a-bfad-44e9c7cf3b4f None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:11.939092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca14a5-6a63-4a9a-bfad-44e9c7cf3b4f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:11.939092 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca14a5-6a63-4a9a-bfad-44e9c7cf3b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:11.970322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-27ca14a5-6a63-4a9a-bfad-44e9c7cf3b4f None None] MFA Rules not processed for user `f30ea147cc194c2f84add802a6ff624f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:11.971579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-27ca14a5-6a63-4a9a-bfad-44e9c7cf3b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:12.042503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1306/2624] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:12.053455 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7feb927-c3c1-4051-88ad-da0aca5a4732 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:12.055188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b7feb927-c3c1-4051-88ad-da0aca5a4732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:12.118410 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7feb927-c3c1-4051-88ad-da0aca5a4732 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:12.122986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234135f2-b407-4d08-9e42-4f9bc62b2cf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:12.123625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234135f2-b407-4d08-9e42-4f9bc62b2cf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:12.123903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234135f2-b407-4d08-9e42-4f9bc62b2cf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:12.124181 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234135f2-b407-4d08-9e42-4f9bc62b2cf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:12.133250 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234135f2-b407-4d08-9e42-4f9bc62b2cf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:12.134720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234135f2-b407-4d08-9e42-4f9bc62b2cf3 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:12.143952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1319/2625] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:15.127895 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff94984-cef8-49e8-8e43-ced9dc230e5a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:15.128389 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff94984-cef8-49e8-8e43-ced9dc230e5a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:15.128490 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff94984-cef8-49e8-8e43-ced9dc230e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:15.147142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7ff94984-cef8-49e8-8e43-ced9dc230e5a None None] MFA Rules not processed for user `f30ea147cc194c2f84add802a6ff624f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:15.148358 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7ff94984-cef8-49e8-8e43-ced9dc230e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:15.155516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1307/2626] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:19.353905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234135f2-b407-4d08-9e42-4f9bc62b2cf3 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:19.357149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-234135f2-b407-4d08-9e42-4f9bc62b2cf3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:19.414248 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234135f2-b407-4d08-9e42-4f9bc62b2cf3 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:19.418172 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835308d-5612-41cc-a72b-0fdad15b2db0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:19.418807 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835308d-5612-41cc-a72b-0fdad15b2db0 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:19.419095 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835308d-5612-41cc-a72b-0fdad15b2db0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:19.419504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835308d-5612-41cc-a72b-0fdad15b2db0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:19.421500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f835308d-5612-41cc-a72b-0fdad15b2db0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:19.448042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:19.486399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835308d-5612-41cc-a72b-0fdad15b2db0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:19.487772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835308d-5612-41cc-a72b-0fdad15b2db0 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:19.497630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1320/2627] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:24.011073 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff94984-cef8-49e8-8e43-ced9dc230e5a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:24.012579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7ff94984-cef8-49e8-8e43-ced9dc230e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:24.028230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff94984-cef8-49e8-8e43-ced9dc230e5a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:24.032784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae8bfb-8532-4c5c-9584-b195670bbebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:24.033465 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8bfb-8532-4c5c-9584-b195670bbebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:24.033745 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8bfb-8532-4c5c-9584-b195670bbebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:24.034019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8bfb-8532-4c5c-9584-b195670bbebe admin admin] PATH_INFO: `/v3/users/ca7d773290a9483d91ebefad341addee` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:24.035182 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae8bfb-8532-4c5c-9584-b195670bbebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca7d773290a9483d91ebefad341addee)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:24.037117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae8bfb-8532-4c5c-9584-b195670bbebe admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:24.049566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:24.186017 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-53ae8bfb-8532-4c5c-9584-b195670bbebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca7d773290a9483d91ebefad341addee'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:24.193342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-53ae8bfb-8532-4c5c-9584-b195670bbebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca7d773290a9483d91ebefad341addee'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:24.255800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-53ae8bfb-8532-4c5c-9584-b195670bbebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca7d773290a9483d91ebefad341addee'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:24.256518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-53ae8bfb-8532-4c5c-9584-b195670bbebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca7d773290a9483d91ebefad341addee'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:24.270695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-53ae8bfb-8532-4c5c-9584-b195670bbebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca7d773290a9483d91ebefad341addee'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:24.278000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1308/2628] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:24 2022] DELETE /identity/v3/users/ca7d773290a9483d91ebefad341addee => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:24.555362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835308d-5612-41cc-a72b-0fdad15b2db0 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:24.567026 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835308d-5612-41cc-a72b-0fdad15b2db0 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:24.624686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:24.625360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:24.625647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:24.625927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] PATH_INFO: `/v3/projects/9e962e97777048b79415ba13adcc95be` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:24.627094 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e962e97777048b79415ba13adcc95be)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:24.629046 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:24.725080 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e962e97777048b79415ba13adcc95be', 'request_id': 'req-494063c1-e9b8-4f02-b731-ebb96491f13a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:24.730702 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e962e97777048b79415ba13adcc95be', 'request_id': 'req-494063c1-e9b8-4f02-b731-ebb96491f13a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:24.736956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e962e97777048b79415ba13adcc95be', 'request_id': 'req-494063c1-e9b8-4f02-b731-ebb96491f13a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:24.807655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] The token cache is being invalidate because project 9e962e97777048b79415ba13adcc95be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:24.807655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:24.807655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-494063c1-e9b8-4f02-b731-ebb96491f13a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:24.808450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1321/2629] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:24 2022] DELETE /identity/v3/projects/9e962e97777048b79415ba13adcc95be => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:24.813361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37a209-c357-46da-9b1d-a2fa5af4e721 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:24.813654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37a209-c357-46da-9b1d-a2fa5af4e721 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:24.813932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37a209-c357-46da-9b1d-a2fa5af4e721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:24.834101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9a37a209-c357-46da-9b1d-a2fa5af4e721 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:24.835329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9a37a209-c357-46da-9b1d-a2fa5af4e721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:24.897635 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1309/2630] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:24.906454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913f0a7-e3f6-4f66-9758-a9d26313401f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:24.906748 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913f0a7-e3f6-4f66-9758-a9d26313401f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:24.907033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913f0a7-e3f6-4f66-9758-a9d26313401f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:24.926971 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4913f0a7-e3f6-4f66-9758-a9d26313401f None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:24.928182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4913f0a7-e3f6-4f66-9758-a9d26313401f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:24.936745 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1322/2631] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:24.940765 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37a209-c357-46da-9b1d-a2fa5af4e721 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:24.950626 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37a209-c357-46da-9b1d-a2fa5af4e721 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:24.955226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47712a-7648-42ac-937f-6caf40ddd684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:24.955833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47712a-7648-42ac-937f-6caf40ddd684 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:24.956116 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47712a-7648-42ac-937f-6caf40ddd684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:24.956393 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47712a-7648-42ac-937f-6caf40ddd684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:24.957148 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47712a-7648-42ac-937f-6caf40ddd684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:24:24.957455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47712a-7648-42ac-937f-6caf40ddd684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:24.959037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47712a-7648-42ac-937f-6caf40ddd684 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:24.970935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1310/2632] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:24:24.975193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913f0a7-e3f6-4f66-9758-a9d26313401f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:24.985158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913f0a7-e3f6-4f66-9758-a9d26313401f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:24.989843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483f223-f444-4ffc-9888-13b7d21ac0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:24.990443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483f223-f444-4ffc-9888-13b7d21ac0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:24.990727 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483f223-f444-4ffc-9888-13b7d21ac0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:24.991000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483f223-f444-4ffc-9888-13b7d21ac0b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:24.991667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483f223-f444-4ffc-9888-13b7d21ac0b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:24.993612 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483f223-f444-4ffc-9888-13b7d21ac0b0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:25.074841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1323/2633] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:24 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:25.078778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47712a-7648-42ac-937f-6caf40ddd684 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:25.089051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47712a-7648-42ac-937f-6caf40ddd684 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:25.144028 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fe52b-1739-4a75-8a7e-ed0302666954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:25.144639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fe52b-1739-4a75-8a7e-ed0302666954 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:25.144925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fe52b-1739-4a75-8a7e-ed0302666954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:25.145235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fe52b-1739-4a75-8a7e-ed0302666954 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:25.145912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fe52b-1739-4a75-8a7e-ed0302666954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:25.147817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fe52b-1739-4a75-8a7e-ed0302666954 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:25.230440 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1311/2634] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:25 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:25.234301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483f223-f444-4ffc-9888-13b7d21ac0b0 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:25.244926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483f223-f444-4ffc-9888-13b7d21ac0b0 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:25.249686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee758a-3489-4497-a7e4-8103e528214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:25.250376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee758a-3489-4497-a7e4-8103e528214b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:25.250660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee758a-3489-4497-a7e4-8103e528214b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:25.250951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee758a-3489-4497-a7e4-8103e528214b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:25.251630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee758a-3489-4497-a7e4-8103e528214b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:25.253237 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee758a-3489-4497-a7e4-8103e528214b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:25.267797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1324/2635] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:25.268006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fe52b-1739-4a75-8a7e-ed0302666954 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:25.274411 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:25.278842 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fe52b-1739-4a75-8a7e-ed0302666954 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:25.283618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb2df5-eb21-4936-afcd-133f56bf969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:25.284220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb2df5-eb21-4936-afcd-133f56bf969c admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:25.284503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb2df5-eb21-4936-afcd-133f56bf969c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:25.284783 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb2df5-eb21-4936-afcd-133f56bf969c admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:25.285510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb2df5-eb21-4936-afcd-133f56bf969c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:25.287086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb2df5-eb21-4936-afcd-133f56bf969c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:25.298656 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1312/2636] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:25.302621 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee758a-3489-4497-a7e4-8103e528214b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:25.313936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee758a-3489-4497-a7e4-8103e528214b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:25.319917 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0459a8-3fa2-4db9-b9dd-f9981a7fff3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:25.320693 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0459a8-3fa2-4db9-b9dd-f9981a7fff3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:25.321101 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0459a8-3fa2-4db9-b9dd-f9981a7fff3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:25.321442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0459a8-3fa2-4db9-b9dd-f9981a7fff3c admin admin] PATH_INFO: `/v3/projects/126babac1b524cba976e42b12c0c978f/users/50ed64353b2a445c9ccff01ffe02f087/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:25.333455 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:25.337476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0459a8-3fa2-4db9-b9dd-f9981a7fff3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=126babac1b524cba976e42b12c0c978f, user_id=50ed64353b2a445c9ccff01ffe02f087, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:25.340649 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0459a8-3fa2-4db9-b9dd-f9981a7fff3c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:25.442857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1325/2637] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:25 2022] PUT /identity/v3/projects/126babac1b524cba976e42b12c0c978f/users/50ed64353b2a445c9ccff01ffe02f087/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:25.453510 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fed018-aad9-4b7c-9d09-305051461201 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:25.454029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fed018-aad9-4b7c-9d09-305051461201 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:25.454891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fed018-aad9-4b7c-9d09-305051461201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:25.488508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c7fed018-aad9-4b7c-9d09-305051461201 None None] MFA Rules not processed for user `50ed64353b2a445c9ccff01ffe02f087`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:25.489836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c7fed018-aad9-4b7c-9d09-305051461201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:25.570370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1313/2638] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:25.584013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0459a8-3fa2-4db9-b9dd-f9981a7fff3c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:25.586199 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ab0459a8-3fa2-4db9-b9dd-f9981a7fff3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:25.668501 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0459a8-3fa2-4db9-b9dd-f9981a7fff3c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:25.674304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24019c42-9e21-4e3b-b8c7-0908d94e63b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:25.674897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24019c42-9e21-4e3b-b8c7-0908d94e63b9 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:25.675176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24019c42-9e21-4e3b-b8c7-0908d94e63b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:25.675449 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24019c42-9e21-4e3b-b8c7-0908d94e63b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:25.683368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24019c42-9e21-4e3b-b8c7-0908d94e63b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:25.684930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24019c42-9e21-4e3b-b8c7-0908d94e63b9 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:25.693688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1326/2639] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:28.623448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fed018-aad9-4b7c-9d09-305051461201 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:28.624920 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c7fed018-aad9-4b7c-9d09-305051461201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:28.682160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fed018-aad9-4b7c-9d09-305051461201 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:28.686450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e67e62-8386-4740-9e76-2cae57139105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:28.687070 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e67e62-8386-4740-9e76-2cae57139105 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:28.687354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e67e62-8386-4740-9e76-2cae57139105 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:28.687629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e67e62-8386-4740-9e76-2cae57139105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:28.689401 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-37e67e62-8386-4740-9e76-2cae57139105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:28.753882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e67e62-8386-4740-9e76-2cae57139105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:28.755253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e67e62-8386-4740-9e76-2cae57139105 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:28.764934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1314/2640] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:35.954048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:37.140007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24019c42-9e21-4e3b-b8c7-0908d94e63b9 service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:37.204191 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24019c42-9e21-4e3b-b8c7-0908d94e63b9 service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:37.209008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:37.209668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:37.209953 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:37.210230 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] PATH_INFO: `/v3/users/50ed64353b2a445c9ccff01ffe02f087` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:37.211763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ed64353b2a445c9ccff01ffe02f087)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:37.213716 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:37.396336 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ed64353b2a445c9ccff01ffe02f087'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:37.397028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ed64353b2a445c9ccff01ffe02f087'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:37.403779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ed64353b2a445c9ccff01ffe02f087'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:37.479497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ed64353b2a445c9ccff01ffe02f087'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:37.480158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ed64353b2a445c9ccff01ffe02f087'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:37.498206 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1327/2641] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:37 2022] DELETE /identity/v3/users/50ed64353b2a445c9ccff01ffe02f087 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:37.502952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43904b-f59d-4efd-b81a-6c14a6c9153b None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:37.503408 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43904b-f59d-4efd-b81a-6c14a6c9153b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:37.503508 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43904b-f59d-4efd-b81a-6c14a6c9153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:37.524008 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7b43904b-f59d-4efd-b81a-6c14a6c9153b None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:37.525237 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7b43904b-f59d-4efd-b81a-6c14a6c9153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:37.553598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:37.589706 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1315/2642] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:37.611940 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:37.613643 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:37.687215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee566a7d-745e-4ff2-9185-981af9348ade admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:37.691893 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee320610-1594-44df-bd7e-677612b88de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:37.692647 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320610-1594-44df-bd7e-677612b88de9 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:37.693042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320610-1594-44df-bd7e-677612b88de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:37.693393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320610-1594-44df-bd7e-677612b88de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:37.705229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee320610-1594-44df-bd7e-677612b88de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:37.706781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee320610-1594-44df-bd7e-677612b88de9 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:37.719240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1328/2643] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:38.475261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43904b-f59d-4efd-b81a-6c14a6c9153b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:38.478509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7b43904b-f59d-4efd-b81a-6c14a6c9153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:38.490771 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43904b-f59d-4efd-b81a-6c14a6c9153b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:38.496584 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:38.497277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:38.497561 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:38.497836 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] PATH_INFO: `/v3/projects/126babac1b524cba976e42b12c0c978f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:38.499103 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=126babac1b524cba976e42b12c0c978f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:38.501023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:38.617243 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '126babac1b524cba976e42b12c0c978f', 'request_id': 'req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:38.624256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '126babac1b524cba976e42b12c0c978f', 'request_id': 'req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:38.702324 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '126babac1b524cba976e42b12c0c978f', 'request_id': 'req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:38.713950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] The token cache is being invalidate because project 126babac1b524cba976e42b12c0c978f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:38.714242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:38.715479 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-bf4e3d1e-a090-4fbf-a400-2973ce2ed6ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:38.717048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1316/2644] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:38 2022] DELETE /identity/v3/projects/126babac1b524cba976e42b12c0c978f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:38.722124 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9714d4a-2b81-4a55-901a-72a5e83349ed None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:38.722596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9714d4a-2b81-4a55-901a-72a5e83349ed None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:38.722700 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9714d4a-2b81-4a55-901a-72a5e83349ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:38.743304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d9714d4a-2b81-4a55-901a-72a5e83349ed None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:38.744582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d9714d4a-2b81-4a55-901a-72a5e83349ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:38.761842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:38.806653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1329/2645] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:38.814608 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c64d10-d241-40ef-9804-86ad1ee761f0 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:38.814906 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c64d10-d241-40ef-9804-86ad1ee761f0 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:38.815189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c64d10-d241-40ef-9804-86ad1ee761f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:38.834651 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-75c64d10-d241-40ef-9804-86ad1ee761f0 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:38.836122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-75c64d10-d241-40ef-9804-86ad1ee761f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:38.844642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1317/2646] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:38.849063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9714d4a-2b81-4a55-901a-72a5e83349ed None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:38.859523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9714d4a-2b81-4a55-901a-72a5e83349ed None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:38.864678 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaa96b-cd67-4c6c-ab86-fa3b263998f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:38.865327 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaa96b-cd67-4c6c-ab86-fa3b263998f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:38.865610 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaa96b-cd67-4c6c-ab86-fa3b263998f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:38.865889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaa96b-cd67-4c6c-ab86-fa3b263998f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:38.866591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaa96b-cd67-4c6c-ab86-fa3b263998f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:24:38.866896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaa96b-cd67-4c6c-ab86-fa3b263998f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:38.868496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaa96b-cd67-4c6c-ab86-fa3b263998f7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:38.879883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1330/2647] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:24:38.884191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c64d10-d241-40ef-9804-86ad1ee761f0 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:38.893509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c64d10-d241-40ef-9804-86ad1ee761f0 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:38.898172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75739065-bd26-407c-b327-fd50397fef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:38.898778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75739065-bd26-407c-b327-fd50397fef89 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:38.899063 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75739065-bd26-407c-b327-fd50397fef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:38.899351 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75739065-bd26-407c-b327-fd50397fef89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:38.900001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75739065-bd26-407c-b327-fd50397fef89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:38.901917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75739065-bd26-407c-b327-fd50397fef89 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:38.991380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1318/2648] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:38 2022] POST /identity/v3/projects => generated 405 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:38.995691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaa96b-cd67-4c6c-ab86-fa3b263998f7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:39.007885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaa96b-cd67-4c6c-ab86-fa3b263998f7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:39.195044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd88a5-0789-4feb-b199-f620a3c4cdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:39.195667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd88a5-0789-4feb-b199-f620a3c4cdd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:39.195954 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd88a5-0789-4feb-b199-f620a3c4cdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:39.196257 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd88a5-0789-4feb-b199-f620a3c4cdd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:39.196958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd88a5-0789-4feb-b199-f620a3c4cdd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:39.199448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd88a5-0789-4feb-b199-f620a3c4cdd1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:39.291302 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1331/2649] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:38 2022] POST /identity/v3/users => generated 303 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:24:39.295558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75739065-bd26-407c-b327-fd50397fef89 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:39.311616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75739065-bd26-407c-b327-fd50397fef89 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:39.316498 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4160401-c07d-4a90-8565-946f25263704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:39.317146 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4160401-c07d-4a90-8565-946f25263704 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:39.319946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4160401-c07d-4a90-8565-946f25263704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:39.319946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4160401-c07d-4a90-8565-946f25263704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:39.319946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4160401-c07d-4a90-8565-946f25263704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:39.320227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4160401-c07d-4a90-8565-946f25263704 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:39.332203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1319/2650] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:39.338459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd88a5-0789-4feb-b199-f620a3c4cdd1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:39.354007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd88a5-0789-4feb-b199-f620a3c4cdd1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:39.361661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fd81e-b5cb-4e62-98cc-b11fcf575447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:39.362079 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fd81e-b5cb-4e62-98cc-b11fcf575447 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:39.362359 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fd81e-b5cb-4e62-98cc-b11fcf575447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:39.362630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fd81e-b5cb-4e62-98cc-b11fcf575447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:39.363339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fd81e-b5cb-4e62-98cc-b11fcf575447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:39.364988 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fd81e-b5cb-4e62-98cc-b11fcf575447 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:39.385122 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1332/2651] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:39.386402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4160401-c07d-4a90-8565-946f25263704 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:39.397448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4160401-c07d-4a90-8565-946f25263704 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:39.402001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e971-e98e-4d3e-b4d3-677b8adc62b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:39.402657 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e971-e98e-4d3e-b4d3-677b8adc62b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:39.402940 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e971-e98e-4d3e-b4d3-677b8adc62b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:39.403301 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e971-e98e-4d3e-b4d3-677b8adc62b5 admin admin] PATH_INFO: `/v3/projects/0033b0b8a5344a67ad0817eebb0508cf/users/7855d48543fe49838283ff6ce5d154db/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:39.428113 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129e971-e98e-4d3e-b4d3-677b8adc62b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0033b0b8a5344a67ad0817eebb0508cf, user_id=7855d48543fe49838283ff6ce5d154db, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:39.430800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129e971-e98e-4d3e-b4d3-677b8adc62b5 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:39.497888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1320/2652] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:39 2022] PUT /identity/v3/projects/0033b0b8a5344a67ad0817eebb0508cf/users/7855d48543fe49838283ff6ce5d154db/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:39.504165 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31e9de-3e7e-4a69-adda-4f5e31b6d0ea None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:39.504455 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31e9de-3e7e-4a69-adda-4f5e31b6d0ea None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:39.504732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31e9de-3e7e-4a69-adda-4f5e31b6d0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:39.528947 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7f31e9de-3e7e-4a69-adda-4f5e31b6d0ea None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:39.528947 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7f31e9de-3e7e-4a69-adda-4f5e31b6d0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:39.583387 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1333/2653] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:39.593905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e971-e98e-4d3e-b4d3-677b8adc62b5 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:39.595659 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2129e971-e98e-4d3e-b4d3-677b8adc62b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:39.628343 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5ea43-8694-4107-b7f2-ee32f5d429bd None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:39.628740 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5ea43-8694-4107-b7f2-ee32f5d429bd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:39.629193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5ea43-8694-4107-b7f2-ee32f5d429bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:39.659614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-92d5ea43-8694-4107-b7f2-ee32f5d429bd None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:39.660916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-92d5ea43-8694-4107-b7f2-ee32f5d429bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:39.662263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e971-e98e-4d3e-b4d3-677b8adc62b5 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:39.666004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3997b2-23e8-48f2-938a-9c045d107d1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:39.666642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3997b2-23e8-48f2-938a-9c045d107d1e service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:39.666913 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3997b2-23e8-48f2-938a-9c045d107d1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:39.667192 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3997b2-23e8-48f2-938a-9c045d107d1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:39.675000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:39.677587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3997b2-23e8-48f2-938a-9c045d107d1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:39.679153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3997b2-23e8-48f2-938a-9c045d107d1e service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:39.687396 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:39.691722 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1321/2654] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:24:39.720040 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1334/2655] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:24:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:39.895224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3997b2-23e8-48f2-938a-9c045d107d1e service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:39.896735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4a3997b2-23e8-48f2-938a-9c045d107d1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:39.979690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3997b2-23e8-48f2-938a-9c045d107d1e service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:39.985304 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c23fe-c27e-48a3-92a9-67a1658ef9ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:39.985921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c23fe-c27e-48a3-92a9-67a1658ef9ff service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:39.986207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c23fe-c27e-48a3-92a9-67a1658ef9ff service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:39.986485 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c23fe-c27e-48a3-92a9-67a1658ef9ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:39.994489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c23fe-c27e-48a3-92a9-67a1658ef9ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:39.996689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c23fe-c27e-48a3-92a9-67a1658ef9ff service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:40.004668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1322/2656] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:24:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:24:40.868860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6e22e-bdc0-4dc1-bc78-e06212be6865 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:40.869348 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6e22e-bdc0-4dc1-bc78-e06212be6865 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:40.869443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6e22e-bdc0-4dc1-bc78-e06212be6865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:40.889066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-49f6e22e-bdc0-4dc1-bc78-e06212be6865 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:40.890296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-49f6e22e-bdc0-4dc1-bc78-e06212be6865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:40.897873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1335/2657] 10.222.0.86 () {58 vars in 967 bytes} [Thu Jul 28 18:24:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:40.910565 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c23fe-c27e-48a3-92a9-67a1658ef9ff service swift] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:40.922060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c23fe-c27e-48a3-92a9-67a1658ef9ff service swift] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:40.927190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6e9c1-f377-4eaa-afd0-6b5b970a31ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:40.927790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6e9c1-f377-4eaa-afd0-6b5b970a31ed service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:40.928098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6e9c1-f377-4eaa-afd0-6b5b970a31ed service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:40.928371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6e9c1-f377-4eaa-afd0-6b5b970a31ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:40.936917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6e9c1-f377-4eaa-afd0-6b5b970a31ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:40.938462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6e9c1-f377-4eaa-afd0-6b5b970a31ed service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:40.946661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1323/2658] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:24:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:24:41.087551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb353c-805f-4f64-aade-4785d1f5f0cf None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:41.087837 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb353c-805f-4f64-aade-4785d1f5f0cf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:41.088115 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb353c-805f-4f64-aade-4785d1f5f0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:41.116911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7ddb353c-805f-4f64-aade-4785d1f5f0cf None None] MFA Rules not processed for user `7855d48543fe49838283ff6ce5d154db`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:41.118185 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7ddb353c-805f-4f64-aade-4785d1f5f0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:41.186521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1336/2659] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:41 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:41.197254 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6e9c1-f377-4eaa-afd0-6b5b970a31ed service swift] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:41.208217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6e9c1-f377-4eaa-afd0-6b5b970a31ed service swift] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:41.213245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b79b34-727a-4b9c-947c-09a50c4afa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:41.213888 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b79b34-727a-4b9c-947c-09a50c4afa8e admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:41.214175 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b79b34-727a-4b9c-947c-09a50c4afa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:41.214471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b79b34-727a-4b9c-947c-09a50c4afa8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:41.215336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b79b34-727a-4b9c-947c-09a50c4afa8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:41.217303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b79b34-727a-4b9c-947c-09a50c4afa8e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:41.304641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1324/2660] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:41 2022] POST /identity/v3/projects => generated 405 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:41.310144 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb353c-805f-4f64-aade-4785d1f5f0cf None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:41.321333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb353c-805f-4f64-aade-4785d1f5f0cf None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:41.379356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f28f44-884a-439a-84f8-d42c198ae181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:41.380004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f28f44-884a-439a-84f8-d42c198ae181 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:41.380297 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f28f44-884a-439a-84f8-d42c198ae181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:41.380625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f28f44-884a-439a-84f8-d42c198ae181 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:41.381376 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f28f44-884a-439a-84f8-d42c198ae181 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:41.383293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f28f44-884a-439a-84f8-d42c198ae181 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:41.461659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1337/2661] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:41 2022] POST /identity/v3/users => generated 309 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:24:41.465794 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b79b34-727a-4b9c-947c-09a50c4afa8e admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:41.476094 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b79b34-727a-4b9c-947c-09a50c4afa8e admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:41.481264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e426d-efc3-4625-a024-519aa1950b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:41.481870 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e426d-efc3-4625-a024-519aa1950b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:41.482151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e426d-efc3-4625-a024-519aa1950b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:41.482424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e426d-efc3-4625-a024-519aa1950b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:41.483125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e426d-efc3-4625-a024-519aa1950b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:41.484735 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e426d-efc3-4625-a024-519aa1950b8a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:41.495132 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1325/2662] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:41.500022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f28f44-884a-439a-84f8-d42c198ae181 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:41.510252 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f28f44-884a-439a-84f8-d42c198ae181 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:41.517139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e576d-b8bf-4bd1-af6f-4e086dd4fc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:41.517843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e576d-b8bf-4bd1-af6f-4e086dd4fc3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:41.518129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e576d-b8bf-4bd1-af6f-4e086dd4fc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:41.518408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e576d-b8bf-4bd1-af6f-4e086dd4fc3f admin admin] PATH_INFO: `/v3/projects/e7e7a68287394d1a86f4ae09cd1c4d99/users/1bc79fc1113642fd96eff59f9d400c18/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:41.534672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e576d-b8bf-4bd1-af6f-4e086dd4fc3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e7a68287394d1a86f4ae09cd1c4d99, user_id=1bc79fc1113642fd96eff59f9d400c18, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:41.537356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e576d-b8bf-4bd1-af6f-4e086dd4fc3f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:41.629075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1338/2663] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:41 2022] PUT /identity/v3/projects/e7e7a68287394d1a86f4ae09cd1c4d99/users/1bc79fc1113642fd96eff59f9d400c18/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:41.633035 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e426d-efc3-4625-a024-519aa1950b8a admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:41.643291 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e426d-efc3-4625-a024-519aa1950b8a admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:41.673820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ba096-30b9-4e93-ba03-16b0b03e96bf None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:41.674204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ba096-30b9-4e93-ba03-16b0b03e96bf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:41.674509 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ba096-30b9-4e93-ba03-16b0b03e96bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:41.694237 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-480ba096-30b9-4e93-ba03-16b0b03e96bf None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:41.695455 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-480ba096-30b9-4e93-ba03-16b0b03e96bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:41.702852 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47156098-fef4-49db-bef2-19cdfbbfbfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:41.703710 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47156098-fef4-49db-bef2-19cdfbbfbfad admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:41.704000 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47156098-fef4-49db-bef2-19cdfbbfbfad admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:41.704285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47156098-fef4-49db-bef2-19cdfbbfbfad admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:41.705012 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47156098-fef4-49db-bef2-19cdfbbfbfad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:41.706778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47156098-fef4-49db-bef2-19cdfbbfbfad admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:41.713053 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:41.717685 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1326/2664] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:41.725361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47156098-fef4-49db-bef2-19cdfbbfbfad admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:41.733009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47156098-fef4-49db-bef2-19cdfbbfbfad admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:41.738271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77813c-0cc4-4367-a558-6c8ab00a1bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:41.738928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77813c-0cc4-4367-a558-6c8ab00a1bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:41.739199 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77813c-0cc4-4367-a558-6c8ab00a1bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:41.739471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77813c-0cc4-4367-a558-6c8ab00a1bd5 admin admin] PATH_INFO: `/v3/projects/e7e7a68287394d1a86f4ae09cd1c4d99/users/1bc79fc1113642fd96eff59f9d400c18/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:41.741576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77813c-0cc4-4367-a558-6c8ab00a1bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e7a68287394d1a86f4ae09cd1c4d99, user_id=1bc79fc1113642fd96eff59f9d400c18, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:41.744337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77813c-0cc4-4367-a558-6c8ab00a1bd5 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:41.754848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1327/2665] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:41 2022] PUT /identity/v3/projects/e7e7a68287394d1a86f4ae09cd1c4d99/users/1bc79fc1113642fd96eff59f9d400c18/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:41.759249 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1339/2666] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:24:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:41.766322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77813c-0cc4-4367-a558-6c8ab00a1bd5 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:41.780988 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:41.835994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77813c-0cc4-4367-a558-6c8ab00a1bd5 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:41.843241 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66640f0-bd4c-4146-903f-919979b32cec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:41.844260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66640f0-bd4c-4146-903f-919979b32cec service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:41.844615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66640f0-bd4c-4146-903f-919979b32cec service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:41.844910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66640f0-bd4c-4146-903f-919979b32cec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:41.902737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66640f0-bd4c-4146-903f-919979b32cec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:41.904342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66640f0-bd4c-4146-903f-919979b32cec service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:41.912778 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1328/2667] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:24:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 18:24:42.555860 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d7000-24c2-4209-97e7-ea1ca4706878 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:42.556355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d7000-24c2-4209-97e7-ea1ca4706878 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:42.556554 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d7000-24c2-4209-97e7-ea1ca4706878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:42.577023 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d53d7000-24c2-4209-97e7-ea1ca4706878 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:42.578315 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d53d7000-24c2-4209-97e7-ea1ca4706878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:42.586052 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1340/2668] 10.222.0.86 () {58 vars in 967 bytes} [Thu Jul 28 18:24:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:42.598426 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66640f0-bd4c-4146-903f-919979b32cec service swift] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:42.609151 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66640f0-bd4c-4146-903f-919979b32cec service swift] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:42.614421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65452b0-b9d0-4bd7-b6b1-13666eb4f32e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:42.615075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65452b0-b9d0-4bd7-b6b1-13666eb4f32e service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:42.615361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65452b0-b9d0-4bd7-b6b1-13666eb4f32e service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:42.615641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65452b0-b9d0-4bd7-b6b1-13666eb4f32e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:42.624987 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65452b0-b9d0-4bd7-b6b1-13666eb4f32e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:42.626631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65452b0-b9d0-4bd7-b6b1-13666eb4f32e service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:42.634974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1329/2669] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:24:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:24:43.183996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cba15-0ae9-4861-8caa-7cb7f2e059d4 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:43.185088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cba15-0ae9-4861-8caa-7cb7f2e059d4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:43.185088 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cba15-0ae9-4861-8caa-7cb7f2e059d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:43.224041 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-624cba15-0ae9-4861-8caa-7cb7f2e059d4 None None] MFA Rules not processed for user `1bc79fc1113642fd96eff59f9d400c18`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:43.225432 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-624cba15-0ae9-4861-8caa-7cb7f2e059d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:43.326955 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1341/2670] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:44.754650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65452b0-b9d0-4bd7-b6b1-13666eb4f32e service swift] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:44.820295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65452b0-b9d0-4bd7-b6b1-13666eb4f32e service swift] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:44.825438 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:44.826071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:44.826355 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:44.826629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] PATH_INFO: `/v3/users/7855d48543fe49838283ff6ce5d154db` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:44.827935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7855d48543fe49838283ff6ce5d154db)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:44.829903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:45.080067 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:45.086073 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7855d48543fe49838283ff6ce5d154db'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.094509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7855d48543fe49838283ff6ce5d154db'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.176473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7855d48543fe49838283ff6ce5d154db'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.177190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7855d48543fe49838283ff6ce5d154db'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.190189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7855d48543fe49838283ff6ce5d154db'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.196095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1330/2671] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:44 2022] DELETE /identity/v3/users/7855d48543fe49838283ff6ce5d154db => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:45.441586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cba15-0ae9-4861-8caa-7cb7f2e059d4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:45.452614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cba15-0ae9-4861-8caa-7cb7f2e059d4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:45.506091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:45.507611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:45.512421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:45.513092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:45.513371 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:45.513645 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] PATH_INFO: `/v3/projects/0033b0b8a5344a67ad0817eebb0508cf` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:45.514887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0033b0b8a5344a67ad0817eebb0508cf)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:45.516763 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:45.520817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af798041-d28e-49d8-b363-8a7516fe1cd4 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:45.525742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc31fb9-bd42-490b-ae11-19df724ef475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:45.526557 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc31fb9-bd42-490b-ae11-19df724ef475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:45.527379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc31fb9-bd42-490b-ae11-19df724ef475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:45.527697 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc31fb9-bd42-490b-ae11-19df724ef475 admin admin] PATH_INFO: `/v3/users/f30ea147cc194c2f84add802a6ff624f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:45.528862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc31fb9-bd42-490b-ae11-19df724ef475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f30ea147cc194c2f84add802a6ff624f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:45.530908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc31fb9-bd42-490b-ae11-19df724ef475 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:45.643582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:45.644810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0033b0b8a5344a67ad0817eebb0508cf', 'request_id': 'req-f622eea5-3655-4e55-8ac7-b55f75107e2f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.650148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0033b0b8a5344a67ad0817eebb0508cf', 'request_id': 'req-f622eea5-3655-4e55-8ac7-b55f75107e2f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.656750 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0033b0b8a5344a67ad0817eebb0508cf', 'request_id': 'req-f622eea5-3655-4e55-8ac7-b55f75107e2f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.726053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9cc31fb9-bd42-490b-ae11-19df724ef475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f30ea147cc194c2f84add802a6ff624f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.732796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9cc31fb9-bd42-490b-ae11-19df724ef475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f30ea147cc194c2f84add802a6ff624f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.740542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] The token cache is being invalidate because project 0033b0b8a5344a67ad0817eebb0508cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:45.740834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.744458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.744458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1342/2672] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:45 2022] DELETE /identity/v3/projects/0033b0b8a5344a67ad0817eebb0508cf => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:45.747619 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:45.748938 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:45.804457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9cc31fb9-bd42-490b-ae11-19df724ef475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f30ea147cc194c2f84add802a6ff624f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.805136 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9cc31fb9-bd42-490b-ae11-19df724ef475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f30ea147cc194c2f84add802a6ff624f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.811523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622eea5-3655-4e55-8ac7-b55f75107e2f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:45.816360 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:45.817034 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:45.817317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:45.817597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] PATH_INFO: `/v3/users/1bc79fc1113642fd96eff59f9d400c18` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:45.818671 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-9cc31fb9-bd42-490b-ae11-19df724ef475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f30ea147cc194c2f84add802a6ff624f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.818857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc79fc1113642fd96eff59f9d400c18)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:45.820919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:45.825322 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1331/2673] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:45 2022] DELETE /identity/v3/users/f30ea147cc194c2f84add802a6ff624f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:45.829767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f46d3-d022-4693-9086-1b11767915bd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:45.830062 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f46d3-d022-4693-9086-1b11767915bd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:45.830337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f46d3-d022-4693-9086-1b11767915bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:45.849923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-be4f46d3-d022-4693-9086-1b11767915bd None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:45.851147 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-be4f46d3-d022-4693-9086-1b11767915bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:45.911255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1332/2674] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:45.921289 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f46d3-d022-4693-9086-1b11767915bd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:45.922677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-be4f46d3-d022-4693-9086-1b11767915bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:45.956135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:45.978692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc79fc1113642fd96eff59f9d400c18'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.979150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f46d3-d022-4693-9086-1b11767915bd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:45.979298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc79fc1113642fd96eff59f9d400c18'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.984214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b9e2a-fb7d-4fa4-ab13-f09fbda298de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:45.984907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b9e2a-fb7d-4fa4-ab13-f09fbda298de service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:45.985223 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b9e2a-fb7d-4fa4-ab13-f09fbda298de service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:45.985620 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b9e2a-fb7d-4fa4-ab13-f09fbda298de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:45.987581 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc79fc1113642fd96eff59f9d400c18'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:45.999808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b9e2a-fb7d-4fa4-ab13-f09fbda298de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:46.001336 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b9e2a-fb7d-4fa4-ab13-f09fbda298de service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:46.010904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1333/2675] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:24:46.039291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc79fc1113642fd96eff59f9d400c18'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.039967 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc79fc1113642fd96eff59f9d400c18'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.060122 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1343/2676] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:45 2022] DELETE /identity/v3/users/1bc79fc1113642fd96eff59f9d400c18 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:46.258245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b9e2a-fb7d-4fa4-ab13-f09fbda298de service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:46.259766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-765b9e2a-fb7d-4fa4-ab13-f09fbda298de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:46.273382 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b9e2a-fb7d-4fa4-ab13-f09fbda298de service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:46.282758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:46.284304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:46.353265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:46.358681 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659ae82-bc0c-4ebf-81e0-9d0ed8958d6e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:46.359501 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:46.360154 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:46.360440 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:46.360720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] PATH_INFO: `/v3/projects/fa13cc57b22c438b842a59ff4e207f10` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:46.361903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa13cc57b22c438b842a59ff4e207f10)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:46.363454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:46.364301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:46.364591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:46.364872 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] PATH_INFO: `/v3/projects/e7e7a68287394d1a86f4ae09cd1c4d99` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:46.365367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:46.366104 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7e7a68287394d1a86f4ae09cd1c4d99)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:46.368000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:46.469054 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa13cc57b22c438b842a59ff4e207f10', 'request_id': 'req-7ceea7e9-2e82-42d9-aa07-98f330d5d662'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.474514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa13cc57b22c438b842a59ff4e207f10', 'request_id': 'req-7ceea7e9-2e82-42d9-aa07-98f330d5d662'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.491707 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7e7a68287394d1a86f4ae09cd1c4d99', 'request_id': 'req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.497140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7e7a68287394d1a86f4ae09cd1c4d99', 'request_id': 'req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.503277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7e7a68287394d1a86f4ae09cd1c4d99', 'request_id': 'req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.581949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa13cc57b22c438b842a59ff4e207f10', 'request_id': 'req-7ceea7e9-2e82-42d9-aa07-98f330d5d662'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.594791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] The token cache is being invalidate because project fa13cc57b22c438b842a59ff4e207f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:46.595084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.595713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7ceea7e9-2e82-42d9-aa07-98f330d5d662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.597279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1334/2677] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:46 2022] DELETE /identity/v3/projects/fa13cc57b22c438b842a59ff4e207f10 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:46.602120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c7eeb-0ce2-4c8c-8de0-4a17934383ba None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:46.602410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c7eeb-0ce2-4c8c-8de0-4a17934383ba None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:46.602694 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c7eeb-0ce2-4c8c-8de0-4a17934383ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:46.621767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e15c7eeb-0ce2-4c8c-8de0-4a17934383ba None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:46.623150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e15c7eeb-0ce2-4c8c-8de0-4a17934383ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:46.643551 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] The token cache is being invalidate because project e7e7a68287394d1a86f4ae09cd1c4d99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:46.650356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.650356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4fbc8a0d-f3d9-4c6d-b6b9-edec4e53399f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:46.650356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1344/2678] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:46 2022] DELETE /identity/v3/projects/e7e7a68287394d1a86f4ae09cd1c4d99 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:46.650865 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f6296-6d15-4543-8458-44410011b228 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:46.651152 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f6296-6d15-4543-8458-44410011b228 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:46.651434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f6296-6d15-4543-8458-44410011b228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:46.672099 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-657f6296-6d15-4543-8458-44410011b228 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:46.673372 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-657f6296-6d15-4543-8458-44410011b228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:46.683500 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1335/2679] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:46.737400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1345/2680] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:46.745306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da2939-d291-468e-8c72-ff1ff56daea4 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:46.745507 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da2939-d291-468e-8c72-ff1ff56daea4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:46.745819 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da2939-d291-468e-8c72-ff1ff56daea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:46.749220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19719a70-966b-4a2d-abd1-a530ca75c0d8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:46.749614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19719a70-966b-4a2d-abd1-a530ca75c0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:46.749888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19719a70-966b-4a2d-abd1-a530ca75c0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:46.755428 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:46.768040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-14da2939-d291-468e-8c72-ff1ff56daea4 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:46.769297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-14da2939-d291-468e-8c72-ff1ff56daea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:46.770777 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-19719a70-966b-4a2d-abd1-a530ca75c0d8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:46.771962 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-19719a70-966b-4a2d-abd1-a530ca75c0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:46.777909 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1336/2681] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:46.780424 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1346/2682] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:46.785065 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da2939-d291-468e-8c72-ff1ff56daea4 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:46.785223 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19719a70-966b-4a2d-abd1-a530ca75c0d8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:46.796480 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da2939-d291-468e-8c72-ff1ff56daea4 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:46.796668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19719a70-966b-4a2d-abd1-a530ca75c0d8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:46.801690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba60dbb-5eed-4148-9ad8-91b1ce012299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:46.810568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53a7dc-aa58-4d83-9061-c469b91c604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:46.810725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba60dbb-5eed-4148-9ad8-91b1ce012299 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:46.810725 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba60dbb-5eed-4148-9ad8-91b1ce012299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:46.811298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a7dc-aa58-4d83-9061-c469b91c604d admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:46.811396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba60dbb-5eed-4148-9ad8-91b1ce012299 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:46.811490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a7dc-aa58-4d83-9061-c469b91c604d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:46.811490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a7dc-aa58-4d83-9061-c469b91c604d admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:46.811690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba60dbb-5eed-4148-9ad8-91b1ce012299 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:24:46.811690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba60dbb-5eed-4148-9ad8-91b1ce012299 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:46.811871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53a7dc-aa58-4d83-9061-c469b91c604d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:24:46.811871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53a7dc-aa58-4d83-9061-c469b91c604d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:46.812042 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba60dbb-5eed-4148-9ad8-91b1ce012299 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:46.812163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53a7dc-aa58-4d83-9061-c469b91c604d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:46.818492 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1337/2683] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:24:46.818722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1347/2684] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 18:24:46.823405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba60dbb-5eed-4148-9ad8-91b1ce012299 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:46.825119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53a7dc-aa58-4d83-9061-c469b91c604d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:46.834830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53a7dc-aa58-4d83-9061-c469b91c604d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:46.835364 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba60dbb-5eed-4148-9ad8-91b1ce012299 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:46.839720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d03120-5ad5-44cb-bf0a-d51ee25ea7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:46.840220 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073b3f9-8f84-462c-8676-791395cede3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:46.840443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d03120-5ad5-44cb-bf0a-d51ee25ea7eb admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:46.840584 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d03120-5ad5-44cb-bf0a-d51ee25ea7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:46.840827 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073b3f9-8f84-462c-8676-791395cede3a admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:46.840929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d03120-5ad5-44cb-bf0a-d51ee25ea7eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:46.841149 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073b3f9-8f84-462c-8676-791395cede3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:46.841429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073b3f9-8f84-462c-8676-791395cede3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:46.841544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d03120-5ad5-44cb-bf0a-d51ee25ea7eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:46.842095 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073b3f9-8f84-462c-8676-791395cede3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:46.843452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d03120-5ad5-44cb-bf0a-d51ee25ea7eb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:46.844016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073b3f9-8f84-462c-8676-791395cede3a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:46.924439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1338/2685] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:46 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:24:46.924775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1348/2686] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:46 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:24:46.925457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073b3f9-8f84-462c-8676-791395cede3a admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:46.928630 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d03120-5ad5-44cb-bf0a-d51ee25ea7eb admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:46.976920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:46.988098 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073b3f9-8f84-462c-8676-791395cede3a admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:46.992952 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec9320-e877-4a63-989a-72e0625b235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:46.993639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec9320-e877-4a63-989a-72e0625b235a admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:46.993916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec9320-e877-4a63-989a-72e0625b235a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:46.994188 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec9320-e877-4a63-989a-72e0625b235a admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:46.994417 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d03120-5ad5-44cb-bf0a-d51ee25ea7eb admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:46.994912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec9320-e877-4a63-989a-72e0625b235a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:46.996899 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec9320-e877-4a63-989a-72e0625b235a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:46.999438 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d49bd-3fff-48f6-bb5b-c6956a8cb595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.000164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d49bd-3fff-48f6-bb5b-c6956a8cb595 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.000420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d49bd-3fff-48f6-bb5b-c6956a8cb595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.000691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d49bd-3fff-48f6-bb5b-c6956a8cb595 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.001386 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d49bd-3fff-48f6-bb5b-c6956a8cb595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.003347 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d49bd-3fff-48f6-bb5b-c6956a8cb595 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.067981 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1339/2687] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:46 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:47.069210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1349/2688] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:46 2022] POST /identity/v3/users => generated 282 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:47.072954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec9320-e877-4a63-989a-72e0625b235a admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.073764 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d49bd-3fff-48f6-bb5b-c6956a8cb595 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.084834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec9320-e877-4a63-989a-72e0625b235a admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.085036 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d49bd-3fff-48f6-bb5b-c6956a8cb595 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.090530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc862b-aa54-44d0-aa4f-f1c6c1d31333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.090772 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764108a8-e097-44c8-8994-836abef230f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.091163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc862b-aa54-44d0-aa4f-f1c6c1d31333 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.091345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764108a8-e097-44c8-8994-836abef230f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.091549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc862b-aa54-44d0-aa4f-f1c6c1d31333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.091700 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764108a8-e097-44c8-8994-836abef230f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.091862 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc862b-aa54-44d0-aa4f-f1c6c1d31333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.092040 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764108a8-e097-44c8-8994-836abef230f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.092389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc862b-aa54-44d0-aa4f-f1c6c1d31333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.092550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764108a8-e097-44c8-8994-836abef230f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.094226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764108a8-e097-44c8-8994-836abef230f2 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.095235 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc862b-aa54-44d0-aa4f-f1c6c1d31333 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.104834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1340/2689] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:24:47.109782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1350/2690] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:24:47.109928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764108a8-e097-44c8-8994-836abef230f2 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.110730 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc862b-aa54-44d0-aa4f-f1c6c1d31333 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.120377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764108a8-e097-44c8-8994-836abef230f2 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.120532 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc862b-aa54-44d0-aa4f-f1c6c1d31333 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.125167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cc4e2-cbb3-4a8c-bc29-7c4004346c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.125314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72496896-bb93-4b34-a843-1341930f06a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.125760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cc4e2-cbb3-4a8c-bc29-7c4004346c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.126019 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72496896-bb93-4b34-a843-1341930f06a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.126122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cc4e2-cbb3-4a8c-bc29-7c4004346c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.126300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cc4e2-cbb3-4a8c-bc29-7c4004346c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.126496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72496896-bb93-4b34-a843-1341930f06a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.126842 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72496896-bb93-4b34-a843-1341930f06a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.126996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cc4e2-cbb3-4a8c-bc29-7c4004346c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.127688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72496896-bb93-4b34-a843-1341930f06a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.128570 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cc4e2-cbb3-4a8c-bc29-7c4004346c64 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.129389 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72496896-bb93-4b34-a843-1341930f06a6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.139333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1341/2691] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:24:47.140288 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1351/2692] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:47.143672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cc4e2-cbb3-4a8c-bc29-7c4004346c64 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.144056 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72496896-bb93-4b34-a843-1341930f06a6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.154061 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cc4e2-cbb3-4a8c-bc29-7c4004346c64 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.154330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72496896-bb93-4b34-a843-1341930f06a6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.158654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a63e2-b67b-4bc4-a13d-3df0796e9f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.159208 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb4744-bf3c-4bc0-82fe-2cac6fef7b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.159350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a63e2-b67b-4bc4-a13d-3df0796e9f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.159573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a63e2-b67b-4bc4-a13d-3df0796e9f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.159855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a63e2-b67b-4bc4-a13d-3df0796e9f16 admin admin] PATH_INFO: `/v3/projects/6f5a984470494d2fb17432261e6d5835/users/8c5399c0e0bc4d238a6d358caa26fc4e/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.160149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb4744-bf3c-4bc0-82fe-2cac6fef7b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.160428 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb4744-bf3c-4bc0-82fe-2cac6fef7b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.160701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb4744-bf3c-4bc0-82fe-2cac6fef7b9b admin admin] PATH_INFO: `/v3/projects/fe865ae79cfc41c1a2668f91e8cf0f70/users/799a2726f3694d39a70cbdaed5257751/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.175554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a63e2-b67b-4bc4-a13d-3df0796e9f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5a984470494d2fb17432261e6d5835, user_id=8c5399c0e0bc4d238a6d358caa26fc4e, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.175832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb4744-bf3c-4bc0-82fe-2cac6fef7b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe865ae79cfc41c1a2668f91e8cf0f70, user_id=799a2726f3694d39a70cbdaed5257751, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.178216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a63e2-b67b-4bc4-a13d-3df0796e9f16 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.178851 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb4744-bf3c-4bc0-82fe-2cac6fef7b9b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.244867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1342/2693] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:47 2022] PUT /identity/v3/projects/6f5a984470494d2fb17432261e6d5835/users/8c5399c0e0bc4d238a6d358caa26fc4e/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:47.256046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739f215-b666-420a-b22a-6a28d1fd9603 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.256275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739f215-b666-420a-b22a-6a28d1fd9603 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.256554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739f215-b666-420a-b22a-6a28d1fd9603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.285246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1739f215-b666-420a-b22a-6a28d1fd9603 None None] MFA Rules not processed for user `8c5399c0e0bc4d238a6d358caa26fc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:47.285421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1352/2694] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:47 2022] PUT /identity/v3/projects/fe865ae79cfc41c1a2668f91e8cf0f70/users/799a2726f3694d39a70cbdaed5257751/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:47.286675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1739f215-b666-420a-b22a-6a28d1fd9603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:47.290350 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139b062-1a58-420d-8058-14b175a420e2 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.290636 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139b062-1a58-420d-8058-14b175a420e2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.291053 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139b062-1a58-420d-8058-14b175a420e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.309307 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e139b062-1a58-420d-8058-14b175a420e2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:47.310530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e139b062-1a58-420d-8058-14b175a420e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:47.344255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:47.353996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1343/2695] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:47 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:47.358429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739f215-b666-420a-b22a-6a28d1fd9603 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.374399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1353/2696] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:47.384090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139b062-1a58-420d-8058-14b175a420e2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.385505 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e139b062-1a58-420d-8058-14b175a420e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:47.422471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739f215-b666-420a-b22a-6a28d1fd9603 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.427334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae7551-a5f6-4674-9998-9d6d2aeb111f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.427958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7551-a5f6-4674-9998-9d6d2aeb111f admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.428244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7551-a5f6-4674-9998-9d6d2aeb111f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.428527 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7551-a5f6-4674-9998-9d6d2aeb111f admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.429230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae7551-a5f6-4674-9998-9d6d2aeb111f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.431185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae7551-a5f6-4674-9998-9d6d2aeb111f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.447964 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139b062-1a58-420d-8058-14b175a420e2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.452435 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9febd-f77d-40f7-8824-efda083132a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.453087 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9febd-f77d-40f7-8824-efda083132a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.453415 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9febd-f77d-40f7-8824-efda083132a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.453709 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9febd-f77d-40f7-8824-efda083132a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.463274 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9febd-f77d-40f7-8824-efda083132a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.465680 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9febd-f77d-40f7-8824-efda083132a5 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.474660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1354/2697] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:24:47.515345 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1344/2698] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:47 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:24:47.519328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9febd-f77d-40f7-8824-efda083132a5 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.529748 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9febd-f77d-40f7-8824-efda083132a5 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.558960 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:47.586469 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390ec4b-f66f-44ad-8dbf-a396081ead6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.587086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390ec4b-f66f-44ad-8dbf-a396081ead6e admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.587369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390ec4b-f66f-44ad-8dbf-a396081ead6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.587660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390ec4b-f66f-44ad-8dbf-a396081ead6e admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.588294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390ec4b-f66f-44ad-8dbf-a396081ead6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.590991 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390ec4b-f66f-44ad-8dbf-a396081ead6e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.722642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1355/2699] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:47 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:47.727049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae7551-a5f6-4674-9998-9d6d2aeb111f admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.738016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae7551-a5f6-4674-9998-9d6d2aeb111f admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.742809 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414332ea-7499-4cf6-a4de-ea364163fd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.743433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414332ea-7499-4cf6-a4de-ea364163fd65 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.743904 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414332ea-7499-4cf6-a4de-ea364163fd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.744211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414332ea-7499-4cf6-a4de-ea364163fd65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.744915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414332ea-7499-4cf6-a4de-ea364163fd65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.747051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414332ea-7499-4cf6-a4de-ea364163fd65 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.758371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1345/2700] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:47.763185 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390ec4b-f66f-44ad-8dbf-a396081ead6e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.774414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390ec4b-f66f-44ad-8dbf-a396081ead6e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.779525 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b095f88-e90a-4323-be6f-4ecdfbb566af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.780284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b095f88-e90a-4323-be6f-4ecdfbb566af admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.780642 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b095f88-e90a-4323-be6f-4ecdfbb566af admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.781026 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b095f88-e90a-4323-be6f-4ecdfbb566af admin admin] PATH_INFO: `/v3/projects/246ce62d820849c79a3885b77b485e08/users/68a087e693fc440384cda28376f80481/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.796625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b095f88-e90a-4323-be6f-4ecdfbb566af admin admin] RBAC: Authorizing `identity:create_grant(project_id=246ce62d820849c79a3885b77b485e08, user_id=68a087e693fc440384cda28376f80481, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.799420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b095f88-e90a-4323-be6f-4ecdfbb566af admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.857379 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1356/2701] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:47 2022] PUT /identity/v3/projects/246ce62d820849c79a3885b77b485e08/users/68a087e693fc440384cda28376f80481/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:47.861702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414332ea-7499-4cf6-a4de-ea364163fd65 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.882357 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414332ea-7499-4cf6-a4de-ea364163fd65 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.938621 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9dade-103b-4160-80bf-3ef2c80597c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.939259 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9dade-103b-4160-80bf-3ef2c80597c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.939548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9dade-103b-4160-80bf-3ef2c80597c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.939833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9dade-103b-4160-80bf-3ef2c80597c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.940530 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9dade-103b-4160-80bf-3ef2c80597c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.942518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9dade-103b-4160-80bf-3ef2c80597c1 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.952546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1346/2702] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:24:47.956817 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b095f88-e90a-4323-be6f-4ecdfbb566af admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:47.968530 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b095f88-e90a-4323-be6f-4ecdfbb566af admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:47.973237 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bec5-9c92-4ef4-9baa-a94628c97762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:47.973900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bec5-9c92-4ef4-9baa-a94628c97762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:47.974177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bec5-9c92-4ef4-9baa-a94628c97762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:47.974452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bec5-9c92-4ef4-9baa-a94628c97762 admin admin] PATH_INFO: `/v3/projects/246ce62d820849c79a3885b77b485e08/users/68a087e693fc440384cda28376f80481/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:47.976421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bec5-9c92-4ef4-9baa-a94628c97762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=246ce62d820849c79a3885b77b485e08, user_id=68a087e693fc440384cda28376f80481, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:47.979044 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bec5-9c92-4ef4-9baa-a94628c97762 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:47.990940 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1357/2703] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:47 2022] PUT /identity/v3/projects/246ce62d820849c79a3885b77b485e08/users/68a087e693fc440384cda28376f80481/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:48.001831 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578a85d-eada-4b73-8cf4-0a466cf23baf None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:48.002126 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578a85d-eada-4b73-8cf4-0a466cf23baf None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:48.002409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578a85d-eada-4b73-8cf4-0a466cf23baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:48.034295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d578a85d-eada-4b73-8cf4-0a466cf23baf None None] MFA Rules not processed for user `68a087e693fc440384cda28376f80481`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:48.034295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d578a85d-eada-4b73-8cf4-0a466cf23baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:48.095267 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:48.120183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1347/2704] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:48 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:48.134369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bec5-9c92-4ef4-9baa-a94628c97762 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:48.136193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0c86bec5-9c92-4ef4-9baa-a94628c97762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:48.194226 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bec5-9c92-4ef4-9baa-a94628c97762 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:48.197918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c877be4-6c32-4d51-90e5-7150dd7ebb1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:48.198545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c877be4-6c32-4d51-90e5-7150dd7ebb1c service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:48.198855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c877be4-6c32-4d51-90e5-7150dd7ebb1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:48.199135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c877be4-6c32-4d51-90e5-7150dd7ebb1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:48.208580 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c877be4-6c32-4d51-90e5-7150dd7ebb1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:48.210403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c877be4-6c32-4d51-90e5-7150dd7ebb1c service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:48.220519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1358/2705] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:48 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:49.143282 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578a85d-eada-4b73-8cf4-0a466cf23baf None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:49.212896 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578a85d-eada-4b73-8cf4-0a466cf23baf None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:49.221246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:49.221906 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:49.222183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:49.222455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] PATH_INFO: `/v3/users/8c5399c0e0bc4d238a6d358caa26fc4e` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:49.223643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c5399c0e0bc4d238a6d358caa26fc4e)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:49.225719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:49.379297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c5399c0e0bc4d238a6d358caa26fc4e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:49.387055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c5399c0e0bc4d238a6d358caa26fc4e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:49.467833 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c5399c0e0bc4d238a6d358caa26fc4e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:49.468562 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c5399c0e0bc4d238a6d358caa26fc4e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:49.490737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c5399c0e0bc4d238a6d358caa26fc4e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:49.505026 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1348/2706] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:49 2022] DELETE /identity/v3/users/8c5399c0e0bc4d238a6d358caa26fc4e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:49.509552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d237-3ada-45b0-9e99-f22a8cf776e3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:49.510039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d237-3ada-45b0-9e99-f22a8cf776e3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:49.510146 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d237-3ada-45b0-9e99-f22a8cf776e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:49.537969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-24e1d237-3ada-45b0-9e99-f22a8cf776e3 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:49.537969 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-24e1d237-3ada-45b0-9e99-f22a8cf776e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:49.604754 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1359/2707] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:49.615942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:49.618270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:49.738184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919023e-26cb-44b8-87fd-fa19a59489cc admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:49.742224 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddbfed-3626-4924-9ab6-223d9c119227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:49.742914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbfed-3626-4924-9ab6-223d9c119227 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:49.743202 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbfed-3626-4924-9ab6-223d9c119227 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:49.743480 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbfed-3626-4924-9ab6-223d9c119227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:49.754112 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbfed-3626-4924-9ab6-223d9c119227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:49.755578 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbfed-3626-4924-9ab6-223d9c119227 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:49.767591 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1349/2708] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:24:50.515881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1d237-3ada-45b0-9e99-f22a8cf776e3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:50.522760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-24e1d237-3ada-45b0-9e99-f22a8cf776e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:50.540304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1d237-3ada-45b0-9e99-f22a8cf776e3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:50.553619 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:50.554188 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:50.554472 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:50.554758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] PATH_INFO: `/v3/projects/6f5a984470494d2fb17432261e6d5835` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:50.556823 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5a984470494d2fb17432261e6d5835)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:50.559542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:50.595694 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:50.675994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5a984470494d2fb17432261e6d5835', 'request_id': 'req-9143d361-1e06-4999-8cc8-d66e4e27a022'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:50.681316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5a984470494d2fb17432261e6d5835', 'request_id': 'req-9143d361-1e06-4999-8cc8-d66e4e27a022'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:50.687812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5a984470494d2fb17432261e6d5835', 'request_id': 'req-9143d361-1e06-4999-8cc8-d66e4e27a022'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:50.771973 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] The token cache is being invalidate because project 6f5a984470494d2fb17432261e6d5835 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:50.772361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:50.774935 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:50.780517 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1360/2709] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:50 2022] DELETE /identity/v3/projects/6f5a984470494d2fb17432261e6d5835 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:50.781776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddbfed-3626-4924-9ab6-223d9c119227 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:50.798229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddbfed-3626-4924-9ab6-223d9c119227 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:50.809102 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:50.874958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a4d73-896d-4f3a-9171-b298dc6a8a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:50.875628 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a4d73-896d-4f3a-9171-b298dc6a8a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:50.875915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a4d73-896d-4f3a-9171-b298dc6a8a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:50.876198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a4d73-896d-4f3a-9171-b298dc6a8a98 admin admin] PATH_INFO: `/v3/users/68a087e693fc440384cda28376f80481` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:50.877549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a4d73-896d-4f3a-9171-b298dc6a8a98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a087e693fc440384cda28376f80481)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:50.879872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a4d73-896d-4f3a-9171-b298dc6a8a98 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:51.057432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-437a4d73-896d-4f3a-9171-b298dc6a8a98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a087e693fc440384cda28376f80481'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:51.066165 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-437a4d73-896d-4f3a-9171-b298dc6a8a98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a087e693fc440384cda28376f80481'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:51.113499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-437a4d73-896d-4f3a-9171-b298dc6a8a98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a087e693fc440384cda28376f80481'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:51.114130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-437a4d73-896d-4f3a-9171-b298dc6a8a98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a087e693fc440384cda28376f80481'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:51.127333 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-437a4d73-896d-4f3a-9171-b298dc6a8a98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a087e693fc440384cda28376f80481'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:51.141398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1350/2710] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:50 2022] DELETE /identity/v3/users/68a087e693fc440384cda28376f80481 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:51.572436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:51.572436 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:51.590835 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143d361-1e06-4999-8cc8-d66e4e27a022 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:51.660767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:51.661629 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:51.662559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:51.662946 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] PATH_INFO: `/v3/projects/246ce62d820849c79a3885b77b485e08` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:51.665176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=246ce62d820849c79a3885b77b485e08)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:51.668062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:51.762941 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246ce62d820849c79a3885b77b485e08', 'request_id': 'req-d78018f0-cf11-41ba-9add-434a70947896'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:51.769795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246ce62d820849c79a3885b77b485e08', 'request_id': 'req-d78018f0-cf11-41ba-9add-434a70947896'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:51.777849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246ce62d820849c79a3885b77b485e08', 'request_id': 'req-d78018f0-cf11-41ba-9add-434a70947896'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:51.834754 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] The token cache is being invalidate because project 246ce62d820849c79a3885b77b485e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:51.835234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:51.835866 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d78018f0-cf11-41ba-9add-434a70947896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:51.837717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1361/2711] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:51 2022] DELETE /identity/v3/projects/246ce62d820849c79a3885b77b485e08 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:51.843295 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e1eab-86dd-407c-a41a-4b26662ae000 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:51.843810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e1eab-86dd-407c-a41a-4b26662ae000 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:51.843932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e1eab-86dd-407c-a41a-4b26662ae000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:51.864078 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f14e1eab-86dd-407c-a41a-4b26662ae000 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:51.869038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f14e1eab-86dd-407c-a41a-4b26662ae000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:51.931379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1351/2712] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:51.939263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4dbd0-0b98-4672-9cce-ada50b9ce4b8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:51.939591 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4dbd0-0b98-4672-9cce-ada50b9ce4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:51.939816 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4dbd0-0b98-4672-9cce-ada50b9ce4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:51.959864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-64d4dbd0-0b98-4672-9cce-ada50b9ce4b8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:51.961385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-64d4dbd0-0b98-4672-9cce-ada50b9ce4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:51.973108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1362/2713] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:51.976049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e1eab-86dd-407c-a41a-4b26662ae000 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:51.995855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e1eab-86dd-407c-a41a-4b26662ae000 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:52.001262 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0abe8a-0aa2-4a91-97e1-fcbf4b72ab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:52.001894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0abe8a-0aa2-4a91-97e1-fcbf4b72ab3a admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:52.002180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0abe8a-0aa2-4a91-97e1-fcbf4b72ab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:52.002458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0abe8a-0aa2-4a91-97e1-fcbf4b72ab3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:52.003984 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0abe8a-0aa2-4a91-97e1-fcbf4b72ab3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:24:52.004299 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0abe8a-0aa2-4a91-97e1-fcbf4b72ab3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:52.005951 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0abe8a-0aa2-4a91-97e1-fcbf4b72ab3a admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:52.021006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1352/2714] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:24:52.025918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4dbd0-0b98-4672-9cce-ada50b9ce4b8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:52.042175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4dbd0-0b98-4672-9cce-ada50b9ce4b8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:52.046375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1673b9-9ed0-4fa1-9378-286000007b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:52.047333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1673b9-9ed0-4fa1-9378-286000007b3a admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:52.047820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1673b9-9ed0-4fa1-9378-286000007b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:52.048388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1673b9-9ed0-4fa1-9378-286000007b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:52.049560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1673b9-9ed0-4fa1-9378-286000007b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:52.051757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1673b9-9ed0-4fa1-9378-286000007b3a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:52.128278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1363/2715] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:52 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:52.132958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0abe8a-0aa2-4a91-97e1-fcbf4b72ab3a admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:52.148141 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0abe8a-0aa2-4a91-97e1-fcbf4b72ab3a admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:52.177766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:52.229008 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5156f0-cd82-4002-9dc1-aed184380ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:52.229761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5156f0-cd82-4002-9dc1-aed184380ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:52.230423 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5156f0-cd82-4002-9dc1-aed184380ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:52.230707 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5156f0-cd82-4002-9dc1-aed184380ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:52.231494 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5156f0-cd82-4002-9dc1-aed184380ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:52.234111 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5156f0-cd82-4002-9dc1-aed184380ac5 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:52.343850 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1353/2716] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:52 2022] POST /identity/v3/users => generated 290 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:52.349313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1673b9-9ed0-4fa1-9378-286000007b3a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:52.360761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1673b9-9ed0-4fa1-9378-286000007b3a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:52.366059 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e71835-0b50-44ba-8396-e95643a5e1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:52.366791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e71835-0b50-44ba-8396-e95643a5e1ad admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:52.367196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e71835-0b50-44ba-8396-e95643a5e1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:52.367567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e71835-0b50-44ba-8396-e95643a5e1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:52.368370 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e71835-0b50-44ba-8396-e95643a5e1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:52.370140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e71835-0b50-44ba-8396-e95643a5e1ad admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:52.376544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:52.385062 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1364/2717] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:52.387294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5156f0-cd82-4002-9dc1-aed184380ac5 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:52.400575 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5156f0-cd82-4002-9dc1-aed184380ac5 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:52.407861 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131dc74-c046-41f5-8ecb-b107c6faf226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:52.408565 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131dc74-c046-41f5-8ecb-b107c6faf226 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:52.408996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131dc74-c046-41f5-8ecb-b107c6faf226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:52.409337 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131dc74-c046-41f5-8ecb-b107c6faf226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:52.410108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131dc74-c046-41f5-8ecb-b107c6faf226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:52.411906 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131dc74-c046-41f5-8ecb-b107c6faf226 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:52.435680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1354/2718] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:24:52.438929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e71835-0b50-44ba-8396-e95643a5e1ad admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:52.450743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e71835-0b50-44ba-8396-e95643a5e1ad admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:52.458717 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb2121-6b97-4bae-8622-f721b08d98f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:52.459477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2121-6b97-4bae-8622-f721b08d98f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:52.459834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2121-6b97-4bae-8622-f721b08d98f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:52.460701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2121-6b97-4bae-8622-f721b08d98f5 admin admin] PATH_INFO: `/v3/projects/355786b9b2864848b132a2eaef000608/users/ec597c409fc74e48a05a9d7393b181e5/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:52.476596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb2121-6b97-4bae-8622-f721b08d98f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=355786b9b2864848b132a2eaef000608, user_id=ec597c409fc74e48a05a9d7393b181e5, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:52.479542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb2121-6b97-4bae-8622-f721b08d98f5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:52.533898 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1365/2719] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:52 2022] PUT /identity/v3/projects/355786b9b2864848b132a2eaef000608/users/ec597c409fc74e48a05a9d7393b181e5/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:52.538384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1e463-ddeb-4160-929b-b5b609ac6cec None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:52.538384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1e463-ddeb-4160-929b-b5b609ac6cec None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:52.538384 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1e463-ddeb-4160-929b-b5b609ac6cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:52.568075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c9b1e463-ddeb-4160-929b-b5b609ac6cec None None] MFA Rules not processed for user `ec597c409fc74e48a05a9d7393b181e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:52.569353 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c9b1e463-ddeb-4160-929b-b5b609ac6cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:52.646239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1355/2720] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:52.650689 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb2121-6b97-4bae-8622-f721b08d98f5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:52.734382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb2121-6b97-4bae-8622-f721b08d98f5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:52.739593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679be7ab-8321-4f78-a27d-bbebf1bef908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:52.740091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679be7ab-8321-4f78-a27d-bbebf1bef908 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:52.740377 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679be7ab-8321-4f78-a27d-bbebf1bef908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:52.740778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679be7ab-8321-4f78-a27d-bbebf1bef908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:52.741529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679be7ab-8321-4f78-a27d-bbebf1bef908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:52.743493 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679be7ab-8321-4f78-a27d-bbebf1bef908 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:52.877692 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1366/2721] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:52 2022] POST /identity/v3/projects => generated 381 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:52.883381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1e463-ddeb-4160-929b-b5b609ac6cec None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:52.897160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1e463-ddeb-4160-929b-b5b609ac6cec None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:52.976239 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ade922-fc63-4d43-9abb-6e043d70dcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:52.976954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ade922-fc63-4d43-9abb-6e043d70dcf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:52.977275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ade922-fc63-4d43-9abb-6e043d70dcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:52.977603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ade922-fc63-4d43-9abb-6e043d70dcf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:52.978264 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ade922-fc63-4d43-9abb-6e043d70dcf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:52.981087 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ade922-fc63-4d43-9abb-6e043d70dcf7 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.024810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea9407-6f17-4f36-beb3-6f62c09b3aae None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.025632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea9407-6f17-4f36-beb3-6f62c09b3aae None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.025927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea9407-6f17-4f36-beb3-6f62c09b3aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.062832 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d5ea9407-6f17-4f36-beb3-6f62c09b3aae None None] MFA Rules not processed for user `799a2726f3694d39a70cbdaed5257751`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:53.064091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d5ea9407-6f17-4f36-beb3-6f62c09b3aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:53.099271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1356/2722] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:52 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:53.103207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ade922-fc63-4d43-9abb-6e043d70dcf7 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:53.115803 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ade922-fc63-4d43-9abb-6e043d70dcf7 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:53.121903 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f21bf1-0602-45e7-8d92-a324ee593180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:53.122650 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f21bf1-0602-45e7-8d92-a324ee593180 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.123010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f21bf1-0602-45e7-8d92-a324ee593180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.123450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f21bf1-0602-45e7-8d92-a324ee593180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.124374 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f21bf1-0602-45e7-8d92-a324ee593180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:53.126183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f21bf1-0602-45e7-8d92-a324ee593180 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.137180 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1357/2723] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:24:53.141122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f21bf1-0602-45e7-8d92-a324ee593180 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:53.147141 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1367/2724] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:53.151365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea9407-6f17-4f36-beb3-6f62c09b3aae None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:53.153560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d5ea9407-6f17-4f36-beb3-6f62c09b3aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:53.156055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f21bf1-0602-45e7-8d92-a324ee593180 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:53.162851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce97f9-137f-4abb-bed1-9c41087cda5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:53.163546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce97f9-137f-4abb-bed1-9c41087cda5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.163828 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce97f9-137f-4abb-bed1-9c41087cda5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.164110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce97f9-137f-4abb-bed1-9c41087cda5d admin admin] PATH_INFO: `/v3/projects/a665f7da2ccd42b6bf2daee4ab420bae/users/31c0548ff187431fac2994406a17f38f/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.166886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:53.172655 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea9407-6f17-4f36-beb3-6f62c09b3aae None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:53.178381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8d148-3a57-41aa-8ce5-5b0fd0a975d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:53.178381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8d148-3a57-41aa-8ce5-5b0fd0a975d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.178663 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8d148-3a57-41aa-8ce5-5b0fd0a975d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.178767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8d148-3a57-41aa-8ce5-5b0fd0a975d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.179404 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8d148-3a57-41aa-8ce5-5b0fd0a975d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:53.181330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8d148-3a57-41aa-8ce5-5b0fd0a975d0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.183692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce97f9-137f-4abb-bed1-9c41087cda5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a665f7da2ccd42b6bf2daee4ab420bae, user_id=31c0548ff187431fac2994406a17f38f, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:53.186634 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce97f9-137f-4abb-bed1-9c41087cda5d admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.257229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1358/2725] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:53 2022] PUT /identity/v3/projects/a665f7da2ccd42b6bf2daee4ab420bae/users/31c0548ff187431fac2994406a17f38f/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:53.261653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce97f9-137f-4abb-bed1-9c41087cda5d admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:53.289052 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1368/2726] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:53 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:24:53.290298 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8d148-3a57-41aa-8ce5-5b0fd0a975d0 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:53.343173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:53.354583 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce97f9-137f-4abb-bed1-9c41087cda5d admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:53.378333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8d148-3a57-41aa-8ce5-5b0fd0a975d0 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:53.385097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012cf3d4-35bb-406c-b436-e61bac2c598d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:53.385745 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cf3d4-35bb-406c-b436-e61bac2c598d admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.386564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cf3d4-35bb-406c-b436-e61bac2c598d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.386852 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cf3d4-35bb-406c-b436-e61bac2c598d admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.387593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cf3d4-35bb-406c-b436-e61bac2c598d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:53.389644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cf3d4-35bb-406c-b436-e61bac2c598d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.431729 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50c778-bc46-4890-8554-73bfec777b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:53.432450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50c778-bc46-4890-8554-73bfec777b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.432806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50c778-bc46-4890-8554-73bfec777b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.433221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50c778-bc46-4890-8554-73bfec777b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.434673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50c778-bc46-4890-8554-73bfec777b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:53.435916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50c778-bc46-4890-8554-73bfec777b3e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.447709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1359/2727] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:24:53.452082 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50c778-bc46-4890-8554-73bfec777b3e admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:53.462869 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50c778-bc46-4890-8554-73bfec777b3e admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:53.468242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793cf38-ce55-484a-b008-05f9f8955718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:53.468893 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793cf38-ce55-484a-b008-05f9f8955718 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.469235 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793cf38-ce55-484a-b008-05f9f8955718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.469504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793cf38-ce55-484a-b008-05f9f8955718 admin admin] PATH_INFO: `/v3/projects/a665f7da2ccd42b6bf2daee4ab420bae/users/31c0548ff187431fac2994406a17f38f/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.471516 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793cf38-ce55-484a-b008-05f9f8955718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a665f7da2ccd42b6bf2daee4ab420bae, user_id=31c0548ff187431fac2994406a17f38f, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:53.474320 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793cf38-ce55-484a-b008-05f9f8955718 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.486194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1360/2728] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:53 2022] PUT /identity/v3/projects/a665f7da2ccd42b6bf2daee4ab420bae/users/31c0548ff187431fac2994406a17f38f/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:53.491464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1369/2729] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:53 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:24:53.496548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d33bc-8929-4912-9375-2aa46c795ab2 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.496924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d33bc-8929-4912-9375-2aa46c795ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.497305 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d33bc-8929-4912-9375-2aa46c795ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.498512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793cf38-ce55-484a-b008-05f9f8955718 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:53.539375 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-052d33bc-8929-4912-9375-2aa46c795ab2 None None] MFA Rules not processed for user `31c0548ff187431fac2994406a17f38f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:53.540895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-052d33bc-8929-4912-9375-2aa46c795ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:53.579983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793cf38-ce55-484a-b008-05f9f8955718 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:53.580573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43100c4c-7687-4bb3-b64f-96632e8c2ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:53.581456 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43100c4c-7687-4bb3-b64f-96632e8c2ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.581805 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43100c4c-7687-4bb3-b64f-96632e8c2ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.582081 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43100c4c-7687-4bb3-b64f-96632e8c2ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.582882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43100c4c-7687-4bb3-b64f-96632e8c2ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:53.584576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43100c4c-7687-4bb3-b64f-96632e8c2ec4 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.595232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1361/2730] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:24:53.599505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43100c4c-7687-4bb3-b64f-96632e8c2ec4 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:53.609817 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43100c4c-7687-4bb3-b64f-96632e8c2ec4 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:53.614425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f3005-1157-400a-ba0c-913deadd6be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:53.615118 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f3005-1157-400a-ba0c-913deadd6be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.615393 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f3005-1157-400a-ba0c-913deadd6be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.615665 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f3005-1157-400a-ba0c-913deadd6be2 admin admin] PATH_INFO: `/v3/projects/b83a89a10e6c4099907b2998c7a7fefa/users/55deab2889444c8ca686c922c10d44bc/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.618776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:53.619889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1370/2731] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:53.633039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f3005-1157-400a-ba0c-913deadd6be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83a89a10e6c4099907b2998c7a7fefa, user_id=55deab2889444c8ca686c922c10d44bc, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:53.636857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d33bc-8929-4912-9375-2aa46c795ab2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:53.636857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-052d33bc-8929-4912-9375-2aa46c795ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:53.637105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f3005-1157-400a-ba0c-913deadd6be2 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.697051 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d33bc-8929-4912-9375-2aa46c795ab2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:53.701496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902bffd-5f3b-4a47-b44b-c48cbfff0dfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:53.702207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902bffd-5f3b-4a47-b44b-c48cbfff0dfb service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.702563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902bffd-5f3b-4a47-b44b-c48cbfff0dfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.702912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902bffd-5f3b-4a47-b44b-c48cbfff0dfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.712996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902bffd-5f3b-4a47-b44b-c48cbfff0dfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:53.714476 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902bffd-5f3b-4a47-b44b-c48cbfff0dfb service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.724461 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1371/2732] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:24:53.764897 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1362/2733] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:53 2022] PUT /identity/v3/projects/b83a89a10e6c4099907b2998c7a7fefa/users/55deab2889444c8ca686c922c10d44bc/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:53.768948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902bffd-5f3b-4a47-b44b-c48cbfff0dfb service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:53.779526 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902bffd-5f3b-4a47-b44b-c48cbfff0dfb service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:53.822334 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:53.836499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2f5c7-3b2a-47d6-ba49-2fb0b743739c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:53.837166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2f5c7-3b2a-47d6-ba49-2fb0b743739c admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.837544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2f5c7-3b2a-47d6-ba49-2fb0b743739c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.837829 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2f5c7-3b2a-47d6-ba49-2fb0b743739c admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.838522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2f5c7-3b2a-47d6-ba49-2fb0b743739c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:53.840162 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2f5c7-3b2a-47d6-ba49-2fb0b743739c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.859907 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1372/2734] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:53.865046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f3005-1157-400a-ba0c-913deadd6be2 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:53.876926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f3005-1157-400a-ba0c-913deadd6be2 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:53.883748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092aa2e-32d8-4e66-bb4e-9e5ed38bd0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:53.884793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092aa2e-32d8-4e66-bb4e-9e5ed38bd0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:53.884793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092aa2e-32d8-4e66-bb4e-9e5ed38bd0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:53.885050 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092aa2e-32d8-4e66-bb4e-9e5ed38bd0d3 admin admin] PATH_INFO: `/v3/projects/b83a89a10e6c4099907b2998c7a7fefa/users/55deab2889444c8ca686c922c10d44bc/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:53.893330 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092aa2e-32d8-4e66-bb4e-9e5ed38bd0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83a89a10e6c4099907b2998c7a7fefa, user_id=55deab2889444c8ca686c922c10d44bc, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:53.896059 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092aa2e-32d8-4e66-bb4e-9e5ed38bd0d3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:53.907458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1363/2735] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:53 2022] PUT /identity/v3/projects/b83a89a10e6c4099907b2998c7a7fefa/users/55deab2889444c8ca686c922c10d44bc/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:54.326909 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2f5c7-3b2a-47d6-ba49-2fb0b743739c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:54.338153 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2f5c7-3b2a-47d6-ba49-2fb0b743739c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:54.405589 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:54.406304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:54.406579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:54.406888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] PATH_INFO: `/v3/users/ec597c409fc74e48a05a9d7393b181e5` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:54.408107 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec597c409fc74e48a05a9d7393b181e5)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:54.410069 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:54.624406 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec597c409fc74e48a05a9d7393b181e5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:54.625143 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec597c409fc74e48a05a9d7393b181e5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:54.633074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec597c409fc74e48a05a9d7393b181e5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:54.702085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec597c409fc74e48a05a9d7393b181e5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:54.703164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec597c409fc74e48a05a9d7393b181e5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:54.722948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1373/2736] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:54 2022] DELETE /identity/v3/users/ec597c409fc74e48a05a9d7393b181e5 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:54.727691 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c80d8d-69cd-4870-9b77-1b681fb35630 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:54.727997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c80d8d-69cd-4870-9b77-1b681fb35630 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:54.728270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c80d8d-69cd-4870-9b77-1b681fb35630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:54.748493 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-22c80d8d-69cd-4870-9b77-1b681fb35630 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:54.749766 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-22c80d8d-69cd-4870-9b77-1b681fb35630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:54.813255 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1364/2737] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:54.823669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:54.826369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:54.892508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6c286-622a-460e-8452-253f945f02f8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:54.896016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57253a-2a87-481a-b57f-eb207dc2022a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:54.896659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57253a-2a87-481a-b57f-eb207dc2022a service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:54.896942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57253a-2a87-481a-b57f-eb207dc2022a service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:54.897247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57253a-2a87-481a-b57f-eb207dc2022a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:54.907583 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57253a-2a87-481a-b57f-eb207dc2022a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:54.909064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57253a-2a87-481a-b57f-eb207dc2022a service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:54.919529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1374/2738] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:55.570198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c80d8d-69cd-4870-9b77-1b681fb35630 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:55.571812 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-22c80d8d-69cd-4870-9b77-1b681fb35630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:55.595790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c80d8d-69cd-4870-9b77-1b681fb35630 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:55.602266 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:55.603222 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:55.603551 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:55.603930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] PATH_INFO: `/v3/projects/355786b9b2864848b132a2eaef000608` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:55.605368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=355786b9b2864848b132a2eaef000608)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:55.609566 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:55.733179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '355786b9b2864848b132a2eaef000608', 'request_id': 'req-e0a9e564-2672-49e7-bef2-e2a50e2316a8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:55.739807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '355786b9b2864848b132a2eaef000608', 'request_id': 'req-e0a9e564-2672-49e7-bef2-e2a50e2316a8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:55.878228 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '355786b9b2864848b132a2eaef000608', 'request_id': 'req-e0a9e564-2672-49e7-bef2-e2a50e2316a8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:55.898653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] The token cache is being invalidate because project 355786b9b2864848b132a2eaef000608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:55.899036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:55.902974 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:55.904926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1365/2739] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:55 2022] DELETE /identity/v3/projects/355786b9b2864848b132a2eaef000608 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:55.909968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57253a-2a87-481a-b57f-eb207dc2022a service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:55.923745 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57253a-2a87-481a-b57f-eb207dc2022a service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:55.954926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:56.057385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5686cac-27ca-432b-8a1d-a8c5d8a5b904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:56.058392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5686cac-27ca-432b-8a1d-a8c5d8a5b904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:56.058849 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5686cac-27ca-432b-8a1d-a8c5d8a5b904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:56.059287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5686cac-27ca-432b-8a1d-a8c5d8a5b904 admin admin] PATH_INFO: `/v3/users/31c0548ff187431fac2994406a17f38f` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:56.060778 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5686cac-27ca-432b-8a1d-a8c5d8a5b904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c0548ff187431fac2994406a17f38f)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:56.062944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5686cac-27ca-432b-8a1d-a8c5d8a5b904 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:56.240269 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c5686cac-27ca-432b-8a1d-a8c5d8a5b904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c0548ff187431fac2994406a17f38f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:56.241400 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c5686cac-27ca-432b-8a1d-a8c5d8a5b904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c0548ff187431fac2994406a17f38f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:56.249904 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c5686cac-27ca-432b-8a1d-a8c5d8a5b904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c0548ff187431fac2994406a17f38f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:56.341220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c5686cac-27ca-432b-8a1d-a8c5d8a5b904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c0548ff187431fac2994406a17f38f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:56.341854 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c5686cac-27ca-432b-8a1d-a8c5d8a5b904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c0548ff187431fac2994406a17f38f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:56.371072 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1375/2740] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:24:55 2022] DELETE /identity/v3/users/31c0548ff187431fac2994406a17f38f => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:56.927934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:56.928488 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:56.944226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9e564-2672-49e7-bef2-e2a50e2316a8 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:57.020303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:57.035702 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:57.036372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:57.036658 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:57.036937 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] PATH_INFO: `/v3/projects/a665f7da2ccd42b6bf2daee4ab420bae` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:57.038427 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a665f7da2ccd42b6bf2daee4ab420bae)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:57.040549 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:57.182956 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a665f7da2ccd42b6bf2daee4ab420bae', 'request_id': 'req-849fdf35-e872-4392-aaac-95bdb38b08c7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:57.196491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a665f7da2ccd42b6bf2daee4ab420bae', 'request_id': 'req-849fdf35-e872-4392-aaac-95bdb38b08c7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:57.268245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a665f7da2ccd42b6bf2daee4ab420bae', 'request_id': 'req-849fdf35-e872-4392-aaac-95bdb38b08c7'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:57.302616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] The token cache is being invalidate because project a665f7da2ccd42b6bf2daee4ab420bae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:24:57.302973 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:57.304109 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-849fdf35-e872-4392-aaac-95bdb38b08c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:24:57.306856 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1366/2741] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:24:56 2022] DELETE /identity/v3/projects/a665f7da2ccd42b6bf2daee4ab420bae => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:57.312049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c917f-55ed-4be2-9a88-e337de999290 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:57.312556 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c917f-55ed-4be2-9a88-e337de999290 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:57.312644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c917f-55ed-4be2-9a88-e337de999290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:57.345066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3a7c917f-55ed-4be2-9a88-e337de999290 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:57.351210 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3a7c917f-55ed-4be2-9a88-e337de999290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:57.433405 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1376/2742] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:57.505448 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7305fb-bbf8-4a5a-9984-7152fd61816e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:57.505736 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7305fb-bbf8-4a5a-9984-7152fd61816e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:57.506016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7305fb-bbf8-4a5a-9984-7152fd61816e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:57.538311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0c7305fb-bbf8-4a5a-9984-7152fd61816e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:57.538311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0c7305fb-bbf8-4a5a-9984-7152fd61816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:57.541005 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1367/2743] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:57.546040 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c917f-55ed-4be2-9a88-e337de999290 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:57.566159 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c917f-55ed-4be2-9a88-e337de999290 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:57.573093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83df9b-f9e8-423f-849a-954423b9fa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:57.574271 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83df9b-f9e8-423f-849a-954423b9fa05 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:57.574760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83df9b-f9e8-423f-849a-954423b9fa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:57.575238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83df9b-f9e8-423f-849a-954423b9fa05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:57.576195 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83df9b-f9e8-423f-849a-954423b9fa05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:24:57.576672 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83df9b-f9e8-423f-849a-954423b9fa05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:57.578527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83df9b-f9e8-423f-849a-954423b9fa05 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:57.597157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1377/2744] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:24:57.601894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7305fb-bbf8-4a5a-9984-7152fd61816e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:57.614741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7305fb-bbf8-4a5a-9984-7152fd61816e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:57.619782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4b264-b326-47b8-9d3f-eb99a01cec97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:57.620439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4b264-b326-47b8-9d3f-eb99a01cec97 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:57.620732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4b264-b326-47b8-9d3f-eb99a01cec97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:57.621019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4b264-b326-47b8-9d3f-eb99a01cec97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:57.621695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4b264-b326-47b8-9d3f-eb99a01cec97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:57.623603 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4b264-b326-47b8-9d3f-eb99a01cec97 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:57.709672 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1368/2745] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:57 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:57.714588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83df9b-f9e8-423f-849a-954423b9fa05 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:57.725722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83df9b-f9e8-423f-849a-954423b9fa05 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:57.736421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:57.802459 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2db00-8ca1-4e4f-a532-f9e4e1f62f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:57.803033 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2db00-8ca1-4e4f-a532-f9e4e1f62f4d admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:57.803317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2db00-8ca1-4e4f-a532-f9e4e1f62f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:57.803588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2db00-8ca1-4e4f-a532-f9e4e1f62f4d admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:57.804883 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2db00-8ca1-4e4f-a532-f9e4e1f62f4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:57.806903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2db00-8ca1-4e4f-a532-f9e4e1f62f4d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:57.910178 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1378/2746] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:57 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:24:57.914499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4b264-b326-47b8-9d3f-eb99a01cec97 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:57.931368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4b264-b326-47b8-9d3f-eb99a01cec97 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:57.938371 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f1f68-4b8b-4045-ba4a-873338ba74aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:57.939651 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f1f68-4b8b-4045-ba4a-873338ba74aa admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:57.940125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f1f68-4b8b-4045-ba4a-873338ba74aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:57.940552 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f1f68-4b8b-4045-ba4a-873338ba74aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:57.941597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f1f68-4b8b-4045-ba4a-873338ba74aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:57.943253 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f1f68-4b8b-4045-ba4a-873338ba74aa admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:57.957436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1369/2747] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:24:57.960103 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2db00-8ca1-4e4f-a532-f9e4e1f62f4d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:57.973287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2db00-8ca1-4e4f-a532-f9e4e1f62f4d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:57.979312 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bd063-7efe-42bb-a358-cbf8990ba5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:57.981025 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bd063-7efe-42bb-a358-cbf8990ba5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:57.981481 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bd063-7efe-42bb-a358-cbf8990ba5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:57.981936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bd063-7efe-42bb-a358-cbf8990ba5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:57.982785 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bd063-7efe-42bb-a358-cbf8990ba5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:57.984668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bd063-7efe-42bb-a358-cbf8990ba5e1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:58.006668 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1379/2748] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:58.010616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f1f68-4b8b-4045-ba4a-873338ba74aa admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:58.022004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f1f68-4b8b-4045-ba4a-873338ba74aa admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:58.032719 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310a6d2-70c5-4ecb-9b7a-4ef1021024e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:58.033709 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310a6d2-70c5-4ecb-9b7a-4ef1021024e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:58.034246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310a6d2-70c5-4ecb-9b7a-4ef1021024e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:58.034520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310a6d2-70c5-4ecb-9b7a-4ef1021024e5 admin admin] PATH_INFO: `/v3/projects/0846441f3c3643b88be20d30031eca63/users/f9748d3693b24bf297e60fc5fb78236b/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:58.056439 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310a6d2-70c5-4ecb-9b7a-4ef1021024e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0846441f3c3643b88be20d30031eca63, user_id=f9748d3693b24bf297e60fc5fb78236b, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:58.060211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310a6d2-70c5-4ecb-9b7a-4ef1021024e5 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:58.140380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1370/2749] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:58 2022] PUT /identity/v3/projects/0846441f3c3643b88be20d30031eca63/users/f9748d3693b24bf297e60fc5fb78236b/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:58.150714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d33c8-6026-4f7d-aaf8-aaede495ac2d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:58.151011 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d33c8-6026-4f7d-aaf8-aaede495ac2d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:58.151299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d33c8-6026-4f7d-aaf8-aaede495ac2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:58.182239 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-373d33c8-6026-4f7d-aaf8-aaede495ac2d None None] MFA Rules not processed for user `f9748d3693b24bf297e60fc5fb78236b`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:58.183535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-373d33c8-6026-4f7d-aaf8-aaede495ac2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:58.267310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1380/2750] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:24:58.271587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310a6d2-70c5-4ecb-9b7a-4ef1021024e5 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:58.346964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310a6d2-70c5-4ecb-9b7a-4ef1021024e5 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:58.352203 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a573210-c951-421e-87e0-a884f0a77e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:58.352834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a573210-c951-421e-87e0-a884f0a77e5c admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:58.353622 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a573210-c951-421e-87e0-a884f0a77e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:58.353829 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a573210-c951-421e-87e0-a884f0a77e5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:58.355434 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a573210-c951-421e-87e0-a884f0a77e5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:58.356592 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a573210-c951-421e-87e0-a884f0a77e5c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:58.447983 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1371/2751] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:24:58 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:58.454855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d33c8-6026-4f7d-aaf8-aaede495ac2d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:58.468335 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d33c8-6026-4f7d-aaf8-aaede495ac2d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:58.538735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f38ea-72f9-4a20-ad4d-ee88d6b9cf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:58.539654 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f38ea-72f9-4a20-ad4d-ee88d6b9cf26 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:58.539938 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f38ea-72f9-4a20-ad4d-ee88d6b9cf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:58.540211 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f38ea-72f9-4a20-ad4d-ee88d6b9cf26 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:58.540888 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f38ea-72f9-4a20-ad4d-ee88d6b9cf26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:58.543393 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f38ea-72f9-4a20-ad4d-ee88d6b9cf26 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:58.648997 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1381/2752] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:24:58 2022] POST /identity/v3/users => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:24:58.651232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a573210-c951-421e-87e0-a884f0a77e5c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:58.662692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a573210-c951-421e-87e0-a884f0a77e5c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:58.667782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baca959-bb1c-43fd-8162-34dd507052f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:58.668400 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baca959-bb1c-43fd-8162-34dd507052f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:58.668682 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baca959-bb1c-43fd-8162-34dd507052f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:58.668958 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baca959-bb1c-43fd-8162-34dd507052f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:58.669686 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baca959-bb1c-43fd-8162-34dd507052f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:58.671538 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baca959-bb1c-43fd-8162-34dd507052f1 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:58.673996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:58.683387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1372/2753] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:58.687795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f38ea-72f9-4a20-ad4d-ee88d6b9cf26 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:58.698867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f38ea-72f9-4a20-ad4d-ee88d6b9cf26 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:58.703765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b852c-cc69-496a-808d-4ad18f127b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:58.704423 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b852c-cc69-496a-808d-4ad18f127b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:58.704701 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b852c-cc69-496a-808d-4ad18f127b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:58.704993 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b852c-cc69-496a-808d-4ad18f127b9d admin admin] PATH_INFO: `/v3/projects/6c8312baf5f24e3e922321be5e724b2d/users/89c10546939743daaebf14174537192f/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:58.708193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:24:58.721916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b852c-cc69-496a-808d-4ad18f127b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c8312baf5f24e3e922321be5e724b2d, user_id=89c10546939743daaebf14174537192f, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:58.724670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b852c-cc69-496a-808d-4ad18f127b9d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:58.846180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1382/2754] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:58 2022] PUT /identity/v3/projects/6c8312baf5f24e3e922321be5e724b2d/users/89c10546939743daaebf14174537192f/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:24:58.850251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baca959-bb1c-43fd-8162-34dd507052f1 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:58.862838 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baca959-bb1c-43fd-8162-34dd507052f1 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:58.954292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f66cab-d7b6-4f51-914c-1e5a6e3abb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:58.955025 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f66cab-d7b6-4f51-914c-1e5a6e3abb04 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:58.955790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f66cab-d7b6-4f51-914c-1e5a6e3abb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:58.956010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f66cab-d7b6-4f51-914c-1e5a6e3abb04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:58.957221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f66cab-d7b6-4f51-914c-1e5a6e3abb04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:58.960818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f66cab-d7b6-4f51-914c-1e5a6e3abb04 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:58.980582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1373/2755] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:24:58.981873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b852c-cc69-496a-808d-4ad18f127b9d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:58.996331 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b852c-cc69-496a-808d-4ad18f127b9d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:59.002884 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83703759-60fc-444d-a536-5afacee37b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:59.003677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83703759-60fc-444d-a536-5afacee37b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:59.003985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83703759-60fc-444d-a536-5afacee37b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:59.004267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83703759-60fc-444d-a536-5afacee37b7e admin admin] PATH_INFO: `/v3/projects/6c8312baf5f24e3e922321be5e724b2d/users/89c10546939743daaebf14174537192f/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:59.007196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83703759-60fc-444d-a536-5afacee37b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c8312baf5f24e3e922321be5e724b2d, user_id=89c10546939743daaebf14174537192f, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:59.010619 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83703759-60fc-444d-a536-5afacee37b7e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:59.025131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1383/2756] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:24:58 2022] PUT /identity/v3/projects/6c8312baf5f24e3e922321be5e724b2d/users/89c10546939743daaebf14174537192f/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:24:59.035879 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477bd3-bfcc-4006-a7d2-7d05e6b7c8ef None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:59.036162 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477bd3-bfcc-4006-a7d2-7d05e6b7c8ef None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:59.036605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477bd3-bfcc-4006-a7d2-7d05e6b7c8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:59.076150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-00477bd3-bfcc-4006-a7d2-7d05e6b7c8ef None None] MFA Rules not processed for user `89c10546939743daaebf14174537192f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:59.077430 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-00477bd3-bfcc-4006-a7d2-7d05e6b7c8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:59.165006 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1374/2757] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:59.176595 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83703759-60fc-444d-a536-5afacee37b7e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:59.178317 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-83703759-60fc-444d-a536-5afacee37b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:59.251215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83703759-60fc-444d-a536-5afacee37b7e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:59.256833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d467a6-9c4f-4f84-b9e0-230b8b0b2132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:59.257560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d467a6-9c4f-4f84-b9e0-230b8b0b2132 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:59.257799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d467a6-9c4f-4f84-b9e0-230b8b0b2132 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:59.258080 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d467a6-9c4f-4f84-b9e0-230b8b0b2132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:59.268601 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d467a6-9c4f-4f84-b9e0-230b8b0b2132 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:59.270084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d467a6-9c4f-4f84-b9e0-230b8b0b2132 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:59.280826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1384/2758] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:24:59.431910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00477bd3-bfcc-4006-a7d2-7d05e6b7c8ef None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:59.443179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00477bd3-bfcc-4006-a7d2-7d05e6b7c8ef None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:24:59.447823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cae637-600a-4e68-8078-697401234345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:24:59.448649 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae637-600a-4e68-8078-697401234345 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:59.449387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae637-600a-4e68-8078-697401234345 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:59.449387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae637-600a-4e68-8078-697401234345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:59.531662 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cae637-600a-4e68-8078-697401234345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:24:59.533292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cae637-600a-4e68-8078-697401234345 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:24:59.544623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1375/2759] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:24:59.816552 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d416b-c8c2-477b-bede-f8f576c4038f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:24:59.817570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d416b-c8c2-477b-bede-f8f576c4038f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:24:59.818043 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d416b-c8c2-477b-bede-f8f576c4038f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:24:59.853698 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c09d416b-c8c2-477b-bede-f8f576c4038f None None] MFA Rules not processed for user `55deab2889444c8ca686c922c10d44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:24:59.855020 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c09d416b-c8c2-477b-bede-f8f576c4038f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:24:59.954827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1385/2760] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:24:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:24:59.971760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cae637-600a-4e68-8078-697401234345 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:24:59.973884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c9cae637-600a-4e68-8078-697401234345 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:00.000723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:00.093432 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cae637-600a-4e68-8078-697401234345 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:00.101946 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf5ab8-21a2-4cd5-8509-b0cd32729c96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47f1552e73648ceb8a421c0543c2974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:00.102791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5ab8-21a2-4cd5-8509-b0cd32729c96 service glance] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:00.103073 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5ab8-21a2-4cd5-8509-b0cd32729c96 service glance] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:00.103539 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5ab8-21a2-4cd5-8509-b0cd32729c96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:00.108155 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c3cf5ab8-21a2-4cd5-8509-b0cd32729c96 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:00.117363 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d416b-c8c2-477b-bede-f8f576c4038f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:00.185925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf5ab8-21a2-4cd5-8509-b0cd32729c96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:00.187588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf5ab8-21a2-4cd5-8509-b0cd32729c96 service glance] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:00.197164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d416b-c8c2-477b-bede-f8f576c4038f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:00.198666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1376/2761] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:24:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:25:00.203272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d60293-47cf-4da3-8bbd-26312a79320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:00.203272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d60293-47cf-4da3-8bbd-26312a79320f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:00.203272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d60293-47cf-4da3-8bbd-26312a79320f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:00.203691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d60293-47cf-4da3-8bbd-26312a79320f admin admin] PATH_INFO: `/v3/users/f9748d3693b24bf297e60fc5fb78236b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:00.204705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d60293-47cf-4da3-8bbd-26312a79320f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9748d3693b24bf297e60fc5fb78236b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:00.206660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d60293-47cf-4da3-8bbd-26312a79320f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:00.208398 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:00.403971 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d1d60293-47cf-4da3-8bbd-26312a79320f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9748d3693b24bf297e60fc5fb78236b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:00.404903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d1d60293-47cf-4da3-8bbd-26312a79320f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9748d3693b24bf297e60fc5fb78236b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:00.414921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d1d60293-47cf-4da3-8bbd-26312a79320f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9748d3693b24bf297e60fc5fb78236b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:00.499532 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c8fe1-6c7b-4be8-9c65-ae9d39b5b81f None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:00.502380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c8fe1-6c7b-4be8-9c65-ae9d39b5b81f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:00.502380 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c8fe1-6c7b-4be8-9c65-ae9d39b5b81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:00.519215 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d1d60293-47cf-4da3-8bbd-26312a79320f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9748d3693b24bf297e60fc5fb78236b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:00.519929 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d1d60293-47cf-4da3-8bbd-26312a79320f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9748d3693b24bf297e60fc5fb78236b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:00.552179 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5d3c8fe1-6c7b-4be8-9c65-ae9d39b5b81f None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:00.553479 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5d3c8fe1-6c7b-4be8-9c65-ae9d39b5b81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:00.571175 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1386/2762] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:25:00 2022] DELETE /identity/v3/users/f9748d3693b24bf297e60fc5fb78236b => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:25:00.573005 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05ae49-c740-40c4-ba8c-99706355b9a2 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:00.573359 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05ae49-c740-40c4-ba8c-99706355b9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:00.573962 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05ae49-c740-40c4-ba8c-99706355b9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:00.598071 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-aa05ae49-c740-40c4-ba8c-99706355b9a2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:00.599836 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-aa05ae49-c740-40c4-ba8c-99706355b9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:00.663588 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1377/2763] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:25:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:00.673846 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c8fe1-6c7b-4be8-9c65-ae9d39b5b81f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:00.675458 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5d3c8fe1-6c7b-4be8-9c65-ae9d39b5b81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:00.687075 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1387/2764] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:00.697499 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05ae49-c740-40c4-ba8c-99706355b9a2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:00.699560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-aa05ae49-c740-40c4-ba8c-99706355b9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:00.741286 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:00.757994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c8fe1-6c7b-4be8-9c65-ae9d39b5b81f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:00.768061 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05ae49-c740-40c4-ba8c-99706355b9a2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:00.772263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fe300-6558-42e8-a3b5-107bb3e3a911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:00.773133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fe300-6558-42e8-a3b5-107bb3e3a911 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:00.773577 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fe300-6558-42e8-a3b5-107bb3e3a911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:00.774031 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fe300-6558-42e8-a3b5-107bb3e3a911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:00.785923 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fe300-6558-42e8-a3b5-107bb3e3a911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:00.787526 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fe300-6558-42e8-a3b5-107bb3e3a911 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:00.799117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1388/2765] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:25:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:25:00.828782 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e16a-20e6-4692-b172-f38be83da199 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:00.829459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e16a-20e6-4692-b172-f38be83da199 service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:00.829740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e16a-20e6-4692-b172-f38be83da199 service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:00.830011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e16a-20e6-4692-b172-f38be83da199 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:00.899027 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e16a-20e6-4692-b172-f38be83da199 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:00.900757 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e16a-20e6-4692-b172-f38be83da199 service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:00.912186 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1378/2766] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:25:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:25:01.275353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fe300-6558-42e8-a3b5-107bb3e3a911 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:01.288448 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-997fe300-6558-42e8-a3b5-107bb3e3a911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:01.322742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fe300-6558-42e8-a3b5-107bb3e3a911 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:01.331795 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:01.333489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:01.333890 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:01.334228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] PATH_INFO: `/v3/projects/0846441f3c3643b88be20d30031eca63` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:01.335986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0846441f3c3643b88be20d30031eca63)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:01.338435 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:01.378936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:01.476319 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0846441f3c3643b88be20d30031eca63', 'request_id': 'req-690b179a-4fb1-43c0-8c00-56d86dc2c328'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:01.486959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0846441f3c3643b88be20d30031eca63', 'request_id': 'req-690b179a-4fb1-43c0-8c00-56d86dc2c328'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:01.493870 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0846441f3c3643b88be20d30031eca63', 'request_id': 'req-690b179a-4fb1-43c0-8c00-56d86dc2c328'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:01.557669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] The token cache is being invalidate because project 0846441f3c3643b88be20d30031eca63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:25:01.558211 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:01.562488 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:01.564739 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1389/2767] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:25:01 2022] DELETE /identity/v3/projects/0846441f3c3643b88be20d30031eca63 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:01.568874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e16a-20e6-4692-b172-f38be83da199 service swift] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:01.582071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e16a-20e6-4692-b172-f38be83da199 service swift] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:01.664851 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d024da-1405-4ede-9de0-89a0d9a5c385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:01.665687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d024da-1405-4ede-9de0-89a0d9a5c385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:01.666039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d024da-1405-4ede-9de0-89a0d9a5c385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:01.666391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d024da-1405-4ede-9de0-89a0d9a5c385 admin admin] PATH_INFO: `/v3/users/89c10546939743daaebf14174537192f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:01.667713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d024da-1405-4ede-9de0-89a0d9a5c385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89c10546939743daaebf14174537192f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:01.670134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d024da-1405-4ede-9de0-89a0d9a5c385 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:01.833713 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:01.842190 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7d024da-1405-4ede-9de0-89a0d9a5c385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89c10546939743daaebf14174537192f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:01.851872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7d024da-1405-4ede-9de0-89a0d9a5c385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89c10546939743daaebf14174537192f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:01.942083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7d024da-1405-4ede-9de0-89a0d9a5c385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89c10546939743daaebf14174537192f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:01.942889 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7d024da-1405-4ede-9de0-89a0d9a5c385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89c10546939743daaebf14174537192f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:01.965714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-c7d024da-1405-4ede-9de0-89a0d9a5c385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89c10546939743daaebf14174537192f'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:01.972724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1379/2768] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:25:01 2022] DELETE /identity/v3/users/89c10546939743daaebf14174537192f => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:02.492356 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:02.493900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:02.508504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b179a-4fb1-43c0-8c00-56d86dc2c328 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:02.592906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:02.593710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:02.594043 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:02.594396 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] PATH_INFO: `/v3/projects/6c8312baf5f24e3e922321be5e724b2d` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:02.595743 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c8312baf5f24e3e922321be5e724b2d)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:02.598161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:02.695186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c8312baf5f24e3e922321be5e724b2d', 'request_id': 'req-76299dbd-a9dd-4195-8977-768f8b010b23'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:02.702293 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c8312baf5f24e3e922321be5e724b2d', 'request_id': 'req-76299dbd-a9dd-4195-8977-768f8b010b23'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:02.712187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c8312baf5f24e3e922321be5e724b2d', 'request_id': 'req-76299dbd-a9dd-4195-8977-768f8b010b23'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:02.759010 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] The token cache is being invalidate because project 6c8312baf5f24e3e922321be5e724b2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:25:02.759333 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:02.760100 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-76299dbd-a9dd-4195-8977-768f8b010b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:02.762550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1390/2769] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:25:02 2022] DELETE /identity/v3/projects/6c8312baf5f24e3e922321be5e724b2d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:02.767334 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2678-0104-4c24-a105-8223f1787599 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:02.768212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2678-0104-4c24-a105-8223f1787599 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:02.768677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2678-0104-4c24-a105-8223f1787599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:02.790269 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-bfcf2678-0104-4c24-a105-8223f1787599 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:02.791500 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-bfcf2678-0104-4c24-a105-8223f1787599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:02.863638 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1380/2770] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:02.872182 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e936361-8f66-4e26-8e3c-3b6635bf5efa None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:02.872474 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e936361-8f66-4e26-8e3c-3b6635bf5efa None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:02.872752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e936361-8f66-4e26-8e3c-3b6635bf5efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:02.896547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0e936361-8f66-4e26-8e3c-3b6635bf5efa None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:02.897927 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0e936361-8f66-4e26-8e3c-3b6635bf5efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:02.907613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1391/2771] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:02.911998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf2678-0104-4c24-a105-8223f1787599 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:02.923641 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf2678-0104-4c24-a105-8223f1787599 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:02.929311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce85fc-24a9-449e-8832-31db1e593138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:02.929932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce85fc-24a9-449e-8832-31db1e593138 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:02.930226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce85fc-24a9-449e-8832-31db1e593138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:02.930514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce85fc-24a9-449e-8832-31db1e593138 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:02.931210 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce85fc-24a9-449e-8832-31db1e593138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:25:02.931572 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce85fc-24a9-449e-8832-31db1e593138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:02.933249 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce85fc-24a9-449e-8832-31db1e593138 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:02.950122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1381/2772] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:25:02.951228 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e936361-8f66-4e26-8e3c-3b6635bf5efa None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:02.961203 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e936361-8f66-4e26-8e3c-3b6635bf5efa None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:02.966544 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acb143-eaaf-4334-84e1-2286b7f35a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:02.967185 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb143-eaaf-4334-84e1-2286b7f35a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:02.967460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb143-eaaf-4334-84e1-2286b7f35a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:02.967735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acb143-eaaf-4334-84e1-2286b7f35a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:02.968402 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acb143-eaaf-4334-84e1-2286b7f35a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:02.970354 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acb143-eaaf-4334-84e1-2286b7f35a0d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:03.067574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1392/2773] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:25:02 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:25:03.071868 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce85fc-24a9-449e-8832-31db1e593138 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:03.083292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce85fc-24a9-449e-8832-31db1e593138 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:03.157135 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fa058-cf6f-4684-b55b-7990775b9f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:03.157774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fa058-cf6f-4684-b55b-7990775b9f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:03.158055 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fa058-cf6f-4684-b55b-7990775b9f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:03.158329 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fa058-cf6f-4684-b55b-7990775b9f1c admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:03.159007 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fa058-cf6f-4684-b55b-7990775b9f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:03.160916 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fa058-cf6f-4684-b55b-7990775b9f1c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:03.252250 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1382/2774] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:25:03 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:25:03.257458 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acb143-eaaf-4334-84e1-2286b7f35a0d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:03.272675 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acb143-eaaf-4334-84e1-2286b7f35a0d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:03.281274 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154243c-9777-4145-aaa0-67833dc93cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:03.282117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154243c-9777-4145-aaa0-67833dc93cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:03.282583 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154243c-9777-4145-aaa0-67833dc93cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:03.283209 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154243c-9777-4145-aaa0-67833dc93cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:03.283996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154243c-9777-4145-aaa0-67833dc93cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:03.286388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154243c-9777-4145-aaa0-67833dc93cd2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:03.289254 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:03.303058 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1393/2775] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:25:03.307038 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fa058-cf6f-4684-b55b-7990775b9f1c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:03.327105 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fa058-cf6f-4684-b55b-7990775b9f1c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:03.336015 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2652c59-fcc0-4103-8bda-d5bc9236d2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:03.336661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2652c59-fcc0-4103-8bda-d5bc9236d2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:03.337091 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2652c59-fcc0-4103-8bda-d5bc9236d2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:03.337377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2652c59-fcc0-4103-8bda-d5bc9236d2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:03.338093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2652c59-fcc0-4103-8bda-d5bc9236d2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:03.339832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2652c59-fcc0-4103-8bda-d5bc9236d2a3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:03.356918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1383/2776] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:25:03.358477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154243c-9777-4145-aaa0-67833dc93cd2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:03.371294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154243c-9777-4145-aaa0-67833dc93cd2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:03.377134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d376e0a-f84c-4b9f-b15d-4906d5e80551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:03.377514 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d376e0a-f84c-4b9f-b15d-4906d5e80551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:03.378073 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d376e0a-f84c-4b9f-b15d-4906d5e80551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:03.378296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d376e0a-f84c-4b9f-b15d-4906d5e80551 admin admin] PATH_INFO: `/v3/projects/9580720476264d42b15584171e57d615/users/5542bb6f6f884ff6865605e2ef5829a3/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:03.404328 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d376e0a-f84c-4b9f-b15d-4906d5e80551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9580720476264d42b15584171e57d615, user_id=5542bb6f6f884ff6865605e2ef5829a3, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:03.407394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d376e0a-f84c-4b9f-b15d-4906d5e80551 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:03.534995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81206a-8d06-4484-8c9f-49ced974d87a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:03.535834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81206a-8d06-4484-8c9f-49ced974d87a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:03.535834 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81206a-8d06-4484-8c9f-49ced974d87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:03.544078 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1394/2777] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:25:03 2022] PUT /identity/v3/projects/9580720476264d42b15584171e57d615/users/5542bb6f6f884ff6865605e2ef5829a3/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:03.593763 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0f81206a-8d06-4484-8c9f-49ced974d87a None None] MFA Rules not processed for user `5542bb6f6f884ff6865605e2ef5829a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:03.595496 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0f81206a-8d06-4484-8c9f-49ced974d87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:03.609995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:03.709560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d376e0a-f84c-4b9f-b15d-4906d5e80551 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:03.710024 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1384/2778] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:03.797157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d376e0a-f84c-4b9f-b15d-4906d5e80551 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:03.802154 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f0ea1-c334-4916-8210-fa9512c0fbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:03.802804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f0ea1-c334-4916-8210-fa9512c0fbbe admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:03.803091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f0ea1-c334-4916-8210-fa9512c0fbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:03.803382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f0ea1-c334-4916-8210-fa9512c0fbbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:03.804054 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f0ea1-c334-4916-8210-fa9512c0fbbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:03.806064 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f0ea1-c334-4916-8210-fa9512c0fbbe admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:03.907548 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1395/2779] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:25:03 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:25:03.911626 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81206a-8d06-4484-8c9f-49ced974d87a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:03.929381 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81206a-8d06-4484-8c9f-49ced974d87a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:04.006011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1c2e2-84b9-4ff5-85ce-a7f1a41af1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:04.006648 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1c2e2-84b9-4ff5-85ce-a7f1a41af1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:04.006935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1c2e2-84b9-4ff5-85ce-a7f1a41af1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:04.007214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1c2e2-84b9-4ff5-85ce-a7f1a41af1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:04.007985 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e1c2e2-84b9-4ff5-85ce-a7f1a41af1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:04.010531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e1c2e2-84b9-4ff5-85ce-a7f1a41af1d0 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:04.578236 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1385/2780] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:25:03 2022] POST /identity/v3/users => generated 293 bytes in 665 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:25:04.579652 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f0ea1-c334-4916-8210-fa9512c0fbbe admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:04.590890 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f0ea1-c334-4916-8210-fa9512c0fbbe admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:04.595992 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f09d5-a4d1-47c4-9644-65513a337e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:04.596951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f09d5-a4d1-47c4-9644-65513a337e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:04.596951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f09d5-a4d1-47c4-9644-65513a337e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:04.597265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f09d5-a4d1-47c4-9644-65513a337e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:04.598108 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f09d5-a4d1-47c4-9644-65513a337e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:04.599848 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f09d5-a4d1-47c4-9644-65513a337e5f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:04.612277 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1396/2781] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:25:04.617359 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1c2e2-84b9-4ff5-85ce-a7f1a41af1d0 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:04.630300 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1c2e2-84b9-4ff5-85ce-a7f1a41af1d0 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:04.636555 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc94a74-65e5-418c-ace4-ff7b83b17636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:04.637441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc94a74-65e5-418c-ace4-ff7b83b17636 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:04.637828 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc94a74-65e5-418c-ace4-ff7b83b17636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:04.638265 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc94a74-65e5-418c-ace4-ff7b83b17636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:04.641758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc94a74-65e5-418c-ace4-ff7b83b17636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:04.643695 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc94a74-65e5-418c-ace4-ff7b83b17636 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:04.670711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1386/2782] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:25:04.671932 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f09d5-a4d1-47c4-9644-65513a337e5f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:04.699705 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f09d5-a4d1-47c4-9644-65513a337e5f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:04.706822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abd37f-ee0b-4841-9e2e-0a0fea4d5d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:04.707924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abd37f-ee0b-4841-9e2e-0a0fea4d5d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:04.708512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abd37f-ee0b-4841-9e2e-0a0fea4d5d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:04.708994 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abd37f-ee0b-4841-9e2e-0a0fea4d5d5e admin admin] PATH_INFO: `/v3/projects/366f35ee65b243faaa7cc7e237ad72e1/users/b4f244a2962546698b4324c57cfba6f8/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:04.733435 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abd37f-ee0b-4841-9e2e-0a0fea4d5d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=366f35ee65b243faaa7cc7e237ad72e1, user_id=b4f244a2962546698b4324c57cfba6f8, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:04.736310 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abd37f-ee0b-4841-9e2e-0a0fea4d5d5e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:04.790497 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1397/2783] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:25:04 2022] PUT /identity/v3/projects/366f35ee65b243faaa7cc7e237ad72e1/users/b4f244a2962546698b4324c57cfba6f8/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:04.802605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823e49a-f21a-40c7-b561-4285da6906e9 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:04.802992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823e49a-f21a-40c7-b561-4285da6906e9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:04.803372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823e49a-f21a-40c7-b561-4285da6906e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:04.840447 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d823e49a-f21a-40c7-b561-4285da6906e9 None None] MFA Rules not processed for user `b4f244a2962546698b4324c57cfba6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:04.845001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d823e49a-f21a-40c7-b561-4285da6906e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:04.940443 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1387/2784] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:04.943608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abd37f-ee0b-4841-9e2e-0a0fea4d5d5e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:05.084138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abd37f-ee0b-4841-9e2e-0a0fea4d5d5e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:05.089762 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae710d08-95e4-457e-9924-70aaac1a0d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:05.090519 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae710d08-95e4-457e-9924-70aaac1a0d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:05.090900 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae710d08-95e4-457e-9924-70aaac1a0d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:05.091284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae710d08-95e4-457e-9924-70aaac1a0d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:05.092157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae710d08-95e4-457e-9924-70aaac1a0d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:05.094280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae710d08-95e4-457e-9924-70aaac1a0d90 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:05.207846 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1398/2785] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:25:04 2022] POST /identity/v3/projects => generated 383 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:25:05.212124 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823e49a-f21a-40c7-b561-4285da6906e9 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:05.226048 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823e49a-f21a-40c7-b561-4285da6906e9 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:05.274950 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:05.351808 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290626d-421e-4658-a56e-7647c1546b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:05.352659 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290626d-421e-4658-a56e-7647c1546b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:05.353079 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290626d-421e-4658-a56e-7647c1546b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:05.354256 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290626d-421e-4658-a56e-7647c1546b54 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:05.355630 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290626d-421e-4658-a56e-7647c1546b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:05.358898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290626d-421e-4658-a56e-7647c1546b54 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:05.499724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1388/2786] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:25:05 2022] POST /identity/v3/users => generated 298 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:25:05.504771 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae710d08-95e4-457e-9924-70aaac1a0d90 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:05.518178 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae710d08-95e4-457e-9924-70aaac1a0d90 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:05.523688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666fd7c-5988-4907-b421-d92e573cf091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:05.524611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666fd7c-5988-4907-b421-d92e573cf091 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:05.524611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666fd7c-5988-4907-b421-d92e573cf091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:05.525022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666fd7c-5988-4907-b421-d92e573cf091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:05.531262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666fd7c-5988-4907-b421-d92e573cf091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:05.532901 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666fd7c-5988-4907-b421-d92e573cf091 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:05.534576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:05.546920 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1399/2787] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:25:05.553963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290626d-421e-4658-a56e-7647c1546b54 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:05.570007 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290626d-421e-4658-a56e-7647c1546b54 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:05.587619 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b2120-e765-4ebd-923f-850f2d6e82f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:05.590835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2120-e765-4ebd-923f-850f2d6e82f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:05.590835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2120-e765-4ebd-923f-850f2d6e82f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:05.590835 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2120-e765-4ebd-923f-850f2d6e82f3 admin admin] PATH_INFO: `/v3/projects/7aa75860ac2f4926825dd2fe4cdb5b31/users/66ff368b3f724665b07bb8db571906e5/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:05.620996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b2120-e765-4ebd-923f-850f2d6e82f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa75860ac2f4926825dd2fe4cdb5b31, user_id=66ff368b3f724665b07bb8db571906e5, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:05.620996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b2120-e765-4ebd-923f-850f2d6e82f3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:05.704009 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1389/2788] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:25:05 2022] PUT /identity/v3/projects/7aa75860ac2f4926825dd2fe4cdb5b31/users/66ff368b3f724665b07bb8db571906e5/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:05.708161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666fd7c-5988-4907-b421-d92e573cf091 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:05.720160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666fd7c-5988-4907-b421-d92e573cf091 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:05.788028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95379908-1a18-473f-9c6b-744de6777c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:05.788756 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95379908-1a18-473f-9c6b-744de6777c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:05.789163 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95379908-1a18-473f-9c6b-744de6777c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:05.789533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95379908-1a18-473f-9c6b-744de6777c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:05.790369 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95379908-1a18-473f-9c6b-744de6777c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:05.792216 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95379908-1a18-473f-9c6b-744de6777c11 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:05.804760 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1400/2789] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:25:05.809318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b2120-e765-4ebd-923f-850f2d6e82f3 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:05.820311 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b2120-e765-4ebd-923f-850f2d6e82f3 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:05.829774 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f336a94-e11a-4c5e-8e92-6489a6b656a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:05.830525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f336a94-e11a-4c5e-8e92-6489a6b656a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:05.831037 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f336a94-e11a-4c5e-8e92-6489a6b656a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:05.831189 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f336a94-e11a-4c5e-8e92-6489a6b656a8 admin admin] PATH_INFO: `/v3/projects/7aa75860ac2f4926825dd2fe4cdb5b31/users/66ff368b3f724665b07bb8db571906e5/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:05.833372 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f336a94-e11a-4c5e-8e92-6489a6b656a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa75860ac2f4926825dd2fe4cdb5b31, user_id=66ff368b3f724665b07bb8db571906e5, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:05.836677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f336a94-e11a-4c5e-8e92-6489a6b656a8 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:05.848997 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1390/2790] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:25:05 2022] PUT /identity/v3/projects/7aa75860ac2f4926825dd2fe4cdb5b31/users/66ff368b3f724665b07bb8db571906e5/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:05.859502 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd6255-a27b-457b-b4c8-97eff0e85538 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:05.859800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd6255-a27b-457b-b4c8-97eff0e85538 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:05.860084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd6255-a27b-457b-b4c8-97eff0e85538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:05.903355 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-48cd6255-a27b-457b-b4c8-97eff0e85538 None None] MFA Rules not processed for user `66ff368b3f724665b07bb8db571906e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:05.904895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-48cd6255-a27b-457b-b4c8-97eff0e85538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:05.989296 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1401/2791] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:05.996425 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f336a94-e11a-4c5e-8e92-6489a6b656a8 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:05.998498 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7f336a94-e11a-4c5e-8e92-6489a6b656a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:06.062921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f336a94-e11a-4c5e-8e92-6489a6b656a8 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:06.067303 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39180ad-b009-460d-97cf-12a6c513ead0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:06.068004 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39180ad-b009-460d-97cf-12a6c513ead0 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:06.068377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39180ad-b009-460d-97cf-12a6c513ead0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:06.068377 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39180ad-b009-460d-97cf-12a6c513ead0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:06.137914 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39180ad-b009-460d-97cf-12a6c513ead0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:06.139535 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39180ad-b009-460d-97cf-12a6c513ead0 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:06.152908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1391/2792] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:25:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:25:09.277311 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd6255-a27b-457b-b4c8-97eff0e85538 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:09.294779 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd6255-a27b-457b-b4c8-97eff0e85538 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:09.298773 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fde5c5-7202-4141-ab2e-7a325460d5b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:09.299921 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fde5c5-7202-4141-ab2e-7a325460d5b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:09.300403 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fde5c5-7202-4141-ab2e-7a325460d5b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:09.300757 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fde5c5-7202-4141-ab2e-7a325460d5b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:09.376653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fde5c5-7202-4141-ab2e-7a325460d5b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:09.378186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fde5c5-7202-4141-ab2e-7a325460d5b3 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:09.388430 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1402/2793] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:25:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:25:11.303783 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39180ad-b009-460d-97cf-12a6c513ead0 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:11.313504 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39180ad-b009-460d-97cf-12a6c513ead0 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:11.317191 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813d351-58e2-4e4e-a593-456ef938e6c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:11.317810 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813d351-58e2-4e4e-a593-456ef938e6c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:11.318096 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813d351-58e2-4e4e-a593-456ef938e6c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:11.318375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813d351-58e2-4e4e-a593-456ef938e6c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:11.327982 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813d351-58e2-4e4e-a593-456ef938e6c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:11.329457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813d351-58e2-4e4e-a593-456ef938e6c9 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:11.339877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1392/2794] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:25:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:25:11.962579 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8766b98-c0f5-422e-ad5b-f8abc7da6f34 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:11.963091 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8766b98-c0f5-422e-ad5b-f8abc7da6f34 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:11.963190 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8766b98-c0f5-422e-ad5b-f8abc7da6f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:11.983325 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b8766b98-c0f5-422e-ad5b-f8abc7da6f34 None None] MFA Rules not processed for user `799a2726f3694d39a70cbdaed5257751`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:11.984527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b8766b98-c0f5-422e-ad5b-f8abc7da6f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:11.993855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:12.039180 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1403/2795] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:16.072273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813d351-58e2-4e4e-a593-456ef938e6c9 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:16.091673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:16.139949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813d351-58e2-4e4e-a593-456ef938e6c9 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:16.144630 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:16.145312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:16.145597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:16.145877 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] PATH_INFO: `/v3/users/5542bb6f6f884ff6865605e2ef5829a3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:16.147060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] RBAC: Authorizing `identity:delete_user(user_id=5542bb6f6f884ff6865605e2ef5829a3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:16.149002 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:16.354915 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5542bb6f6f884ff6865605e2ef5829a3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:16.361806 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5542bb6f6f884ff6865605e2ef5829a3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:16.455631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5542bb6f6f884ff6865605e2ef5829a3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:16.456260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5542bb6f6f884ff6865605e2ef5829a3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:16.470041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5542bb6f6f884ff6865605e2ef5829a3'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:16.476109 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1393/2796] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:25:16 2022] DELETE /identity/v3/users/5542bb6f6f884ff6865605e2ef5829a3 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:16.481144 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf67d4f-8896-4747-ac41-18fd07b9467d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:16.481632 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf67d4f-8896-4747-ac41-18fd07b9467d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:16.481732 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf67d4f-8896-4747-ac41-18fd07b9467d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:16.513830 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9cf67d4f-8896-4747-ac41-18fd07b9467d None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:16.515047 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9cf67d4f-8896-4747-ac41-18fd07b9467d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:16.579829 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1404/2797] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:25:16.586296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:16.588431 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:16.607304 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33428ffc-c230-4c5d-ad1a-3b43021bd0c5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:16.607539 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33428ffc-c230-4c5d-ad1a-3b43021bd0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:16.607912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33428ffc-c230-4c5d-ad1a-3b43021bd0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:16.634911 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-33428ffc-c230-4c5d-ad1a-3b43021bd0c5 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:16.636656 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-33428ffc-c230-4c5d-ad1a-3b43021bd0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:16.662692 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb525ca0-9c31-464e-863a-a384a0c8d2af admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:16.667751 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a409a-50d9-4120-94e7-5811eb809bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:16.668790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a409a-50d9-4120-94e7-5811eb809bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:16.669211 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a409a-50d9-4120-94e7-5811eb809bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:16.669559 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a409a-50d9-4120-94e7-5811eb809bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:16.681582 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a409a-50d9-4120-94e7-5811eb809bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:16.683110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a409a-50d9-4120-94e7-5811eb809bf8 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:16.703200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1394/2798] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:25:16.705123 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1405/2799] 10.222.0.86 () {58 vars in 1009 bytes} [Thu Jul 28 18:25:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 18:25:16.717413 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a409a-50d9-4120-94e7-5811eb809bf8 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:16.718698 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f41a409a-50d9-4120-94e7-5811eb809bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:16.770994 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:16.819936 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a409a-50d9-4120-94e7-5811eb809bf8 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:16.827215 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6adaab-5ae8-40dc-9df5-c697772ad261 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:16.828115 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6adaab-5ae8-40dc-9df5-c697772ad261 service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:16.828115 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6adaab-5ae8-40dc-9df5-c697772ad261 service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:16.828376 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6adaab-5ae8-40dc-9df5-c697772ad261 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:16.839022 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6adaab-5ae8-40dc-9df5-c697772ad261 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:16.840687 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6adaab-5ae8-40dc-9df5-c697772ad261 service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:16.850854 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1395/2800] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:25:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:25:17.306897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33428ffc-c230-4c5d-ad1a-3b43021bd0c5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:17.312022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-33428ffc-c230-4c5d-ad1a-3b43021bd0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:17.336522 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33428ffc-c230-4c5d-ad1a-3b43021bd0c5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:17.376919 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:17.381824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:17.382245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:17.382540 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] PATH_INFO: `/v3/projects/9580720476264d42b15584171e57d615` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:17.388876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9580720476264d42b15584171e57d615)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:17.390930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:17.608875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9580720476264d42b15584171e57d615', 'request_id': 'req-23388bf5-6262-4792-a935-222a4a452ef1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:17.628093 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9580720476264d42b15584171e57d615', 'request_id': 'req-23388bf5-6262-4792-a935-222a4a452ef1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:17.634677 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:17.640374 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9580720476264d42b15584171e57d615', 'request_id': 'req-23388bf5-6262-4792-a935-222a4a452ef1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:17.706070 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] The token cache is being invalidate because project 9580720476264d42b15584171e57d615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:25:17.706362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:17.709083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:17.709864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1406/2801] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:25:17 2022] DELETE /identity/v3/projects/9580720476264d42b15584171e57d615 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:17.714884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6adaab-5ae8-40dc-9df5-c697772ad261 service swift] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:17.733872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6adaab-5ae8-40dc-9df5-c697772ad261 service swift] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:17.843769 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:17.844814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:17.845424 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:17.846115 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] PATH_INFO: `/v3/users/b4f244a2962546698b4324c57cfba6f8` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:17.849558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f244a2962546698b4324c57cfba6f8)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:17.852944 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:18.100935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f244a2962546698b4324c57cfba6f8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:18.109999 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f244a2962546698b4324c57cfba6f8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:18.164184 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f244a2962546698b4324c57cfba6f8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:18.164927 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f244a2962546698b4324c57cfba6f8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:18.190971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f244a2962546698b4324c57cfba6f8'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:18.199715 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1396/2802] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:25:17 2022] DELETE /identity/v3/users/b4f244a2962546698b4324c57cfba6f8 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:25:18.976781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:18.979936 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:18.997148 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23388bf5-6262-4792-a935-222a4a452ef1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:19.086886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:19.087711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:19.088131 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:19.088460 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] PATH_INFO: `/v3/projects/366f35ee65b243faaa7cc7e237ad72e1` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:19.089772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] RBAC: Authorizing `identity:delete_project(project_id=366f35ee65b243faaa7cc7e237ad72e1)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:19.091843 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:19.237686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '366f35ee65b243faaa7cc7e237ad72e1', 'request_id': 'req-f5146929-d461-4d0a-9fa9-9ce078f4961f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:19.248473 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '366f35ee65b243faaa7cc7e237ad72e1', 'request_id': 'req-f5146929-d461-4d0a-9fa9-9ce078f4961f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:19.266563 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '366f35ee65b243faaa7cc7e237ad72e1', 'request_id': 'req-f5146929-d461-4d0a-9fa9-9ce078f4961f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:19.369414 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] The token cache is being invalidate because project 366f35ee65b243faaa7cc7e237ad72e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:25:19.370245 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:19.370870 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:19.372439 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1407/2803] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:25:18 2022] DELETE /identity/v3/projects/366f35ee65b243faaa7cc7e237ad72e1 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:19.376417 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:19.377876 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:19.393444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673682e1-8c84-45d2-9a85-80c1f43228a0 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:19.454882 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:19.467845 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9bc91-7bca-4a2e-934e-e0279ab7da80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:19.468505 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9bc91-7bca-4a2e-934e-e0279ab7da80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:19.468948 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9bc91-7bca-4a2e-934e-e0279ab7da80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:19.469328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9bc91-7bca-4a2e-934e-e0279ab7da80 admin admin] PATH_INFO: `/v3/users/66ff368b3f724665b07bb8db571906e5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:19.471053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9bc91-7bca-4a2e-934e-e0279ab7da80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ff368b3f724665b07bb8db571906e5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:19.473378 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9bc91-7bca-4a2e-934e-e0279ab7da80 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:19.667626 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-88e9bc91-7bca-4a2e-934e-e0279ab7da80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ff368b3f724665b07bb8db571906e5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:19.675540 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-88e9bc91-7bca-4a2e-934e-e0279ab7da80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ff368b3f724665b07bb8db571906e5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:19.747120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-88e9bc91-7bca-4a2e-934e-e0279ab7da80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ff368b3f724665b07bb8db571906e5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:19.747752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-88e9bc91-7bca-4a2e-934e-e0279ab7da80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ff368b3f724665b07bb8db571906e5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:19.760999 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-88e9bc91-7bca-4a2e-934e-e0279ab7da80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ff368b3f724665b07bb8db571906e5'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:19.772518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1397/2804] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:25:19 2022] DELETE /identity/v3/users/66ff368b3f724665b07bb8db571906e5 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:20.264527 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:20.269903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:20.306189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5146929-d461-4d0a-9fa9-9ce078f4961f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:20.386791 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:20.387397 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:20.387686 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:20.387968 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] PATH_INFO: `/v3/projects/7aa75860ac2f4926825dd2fe4cdb5b31` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:20.389433 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa75860ac2f4926825dd2fe4cdb5b31)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:20.391512 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:20.446256 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:20.550450 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa75860ac2f4926825dd2fe4cdb5b31', 'request_id': 'req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:20.556351 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa75860ac2f4926825dd2fe4cdb5b31', 'request_id': 'req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:20.564489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa75860ac2f4926825dd2fe4cdb5b31', 'request_id': 'req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:20.644454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] The token cache is being invalidate because project 7aa75860ac2f4926825dd2fe4cdb5b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:25:20.644774 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:20.645534 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1fadfa06-d87d-48c4-93f7-8ebeaaa68443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:20.647187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1408/2805] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:25:20 2022] DELETE /identity/v3/projects/7aa75860ac2f4926825dd2fe4cdb5b31 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:20.653457 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed151-1b3d-45e8-b29f-2b184711946c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:20.654093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed151-1b3d-45e8-b29f-2b184711946c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:20.654093 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed151-1b3d-45e8-b29f-2b184711946c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:20.675767 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d01ed151-1b3d-45e8-b29f-2b184711946c None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:20.677065 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d01ed151-1b3d-45e8-b29f-2b184711946c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:20.760655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1398/2806] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:20.768834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7fae6-5e72-4473-a1bd-ec6d9fa93737 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:20.769161 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7fae6-5e72-4473-a1bd-ec6d9fa93737 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:20.769444 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7fae6-5e72-4473-a1bd-ec6d9fa93737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:20.793506 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-60b7fae6-5e72-4473-a1bd-ec6d9fa93737 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:20.794962 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-60b7fae6-5e72-4473-a1bd-ec6d9fa93737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:20.803322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1409/2807] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:20.807644 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ed151-1b3d-45e8-b29f-2b184711946c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:20.819404 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ed151-1b3d-45e8-b29f-2b184711946c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:20.824375 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b2a6e-b33b-412a-947c-1c6849c8e6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:20.825045 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b2a6e-b33b-412a-947c-1c6849c8e6ca admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:20.825338 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b2a6e-b33b-412a-947c-1c6849c8e6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:20.825618 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b2a6e-b33b-412a-947c-1c6849c8e6ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:20.826474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b2a6e-b33b-412a-947c-1c6849c8e6ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:25:20.826786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b2a6e-b33b-412a-947c-1c6849c8e6ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:20.828430 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b2a6e-b33b-412a-947c-1c6849c8e6ca admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:20.848290 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7fae6-5e72-4473-a1bd-ec6d9fa93737 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:20.849178 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1399/2808] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:25:20.858399 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7fae6-5e72-4473-a1bd-ec6d9fa93737 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:20.863149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74007282-6ce1-4464-a04f-c7f79b6553ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:20.863765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74007282-6ce1-4464-a04f-c7f79b6553ae admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:20.864045 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74007282-6ce1-4464-a04f-c7f79b6553ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:20.864316 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74007282-6ce1-4464-a04f-c7f79b6553ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:20.865031 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74007282-6ce1-4464-a04f-c7f79b6553ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:20.866912 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74007282-6ce1-4464-a04f-c7f79b6553ae admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:20.952628 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1410/2809] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:25:20 2022] POST /identity/v3/projects => generated 395 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:25:20.956164 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b2a6e-b33b-412a-947c-1c6849c8e6ca admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:20.968647 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b2a6e-b33b-412a-947c-1c6849c8e6ca admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:21.023666 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:21.033412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac98d04-dc06-475c-bdce-4d9546b5d2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:21.036818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac98d04-dc06-475c-bdce-4d9546b5d2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:21.036818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac98d04-dc06-475c-bdce-4d9546b5d2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:21.036818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac98d04-dc06-475c-bdce-4d9546b5d2b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:21.036818 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac98d04-dc06-475c-bdce-4d9546b5d2b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:21.037217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac98d04-dc06-475c-bdce-4d9546b5d2b2 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:21.154398 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1400/2810] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:25:20 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:25:21.159673 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74007282-6ce1-4464-a04f-c7f79b6553ae admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:21.175676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74007282-6ce1-4464-a04f-c7f79b6553ae admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:21.181583 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868a5eb-6cce-471a-a965-d2983a25ab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:21.188958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868a5eb-6cce-471a-a965-d2983a25ab59 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:21.188958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868a5eb-6cce-471a-a965-d2983a25ab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:21.188958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868a5eb-6cce-471a-a965-d2983a25ab59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:21.188958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868a5eb-6cce-471a-a965-d2983a25ab59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:21.188958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868a5eb-6cce-471a-a965-d2983a25ab59 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:21.198945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1411/2811] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:25:21.205589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac98d04-dc06-475c-bdce-4d9546b5d2b2 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:21.224862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac98d04-dc06-475c-bdce-4d9546b5d2b2 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:21.236998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be969d8-4999-4706-9f2a-34e0636e775f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:21.238209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be969d8-4999-4706-9f2a-34e0636e775f admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:21.238668 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be969d8-4999-4706-9f2a-34e0636e775f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:21.239110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be969d8-4999-4706-9f2a-34e0636e775f admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:21.239985 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be969d8-4999-4706-9f2a-34e0636e775f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:21.242704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be969d8-4999-4706-9f2a-34e0636e775f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:21.261361 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868a5eb-6cce-471a-a965-d2983a25ab59 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:21.274185 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868a5eb-6cce-471a-a965-d2983a25ab59 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:21.356579 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1401/2812] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:25:21.358373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f182d5-19bd-4e47-b95e-65e94816fe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:21.358373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f182d5-19bd-4e47-b95e-65e94816fe00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:21.358373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f182d5-19bd-4e47-b95e-65e94816fe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:21.358373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f182d5-19bd-4e47-b95e-65e94816fe00 admin admin] PATH_INFO: `/v3/projects/229ae4f08a1e410496c798a50c043d38/users/01789f5a54314436abbc60f227011827/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:21.358753 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f182d5-19bd-4e47-b95e-65e94816fe00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=229ae4f08a1e410496c798a50c043d38, user_id=01789f5a54314436abbc60f227011827, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:21.358753 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f182d5-19bd-4e47-b95e-65e94816fe00 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:21.503484 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1412/2813] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:25:21 2022] PUT /identity/v3/projects/229ae4f08a1e410496c798a50c043d38/users/01789f5a54314436abbc60f227011827/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:21.507627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126b847-57e6-486e-bbab-414ace6e7400 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:21.508542 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126b847-57e6-486e-bbab-414ace6e7400 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:21.509245 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126b847-57e6-486e-bbab-414ace6e7400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:21.551814 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8126b847-57e6-486e-bbab-414ace6e7400 None None] MFA Rules not processed for user `01789f5a54314436abbc60f227011827`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:21.553354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8126b847-57e6-486e-bbab-414ace6e7400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:21.640654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1402/2814] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:21.654520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f182d5-19bd-4e47-b95e-65e94816fe00 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:21.656278 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-23f182d5-19bd-4e47-b95e-65e94816fe00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:21.715875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f182d5-19bd-4e47-b95e-65e94816fe00 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:21.719674 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b939026-b757-4011-85b0-b49d9c0fd417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:21.720300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b939026-b757-4011-85b0-b49d9c0fd417 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:21.720583 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b939026-b757-4011-85b0-b49d9c0fd417 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:21.720863 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b939026-b757-4011-85b0-b49d9c0fd417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:21.729841 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b939026-b757-4011-85b0-b49d9c0fd417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:21.731291 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b939026-b757-4011-85b0-b49d9c0fd417 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:21.745337 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1413/2815] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:25:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:25:31.637138 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:31.736542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=7,keystone:SELECT=68,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:37.086379 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126b847-57e6-486e-bbab-414ace6e7400 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:37.088154 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8126b847-57e6-486e-bbab-414ace6e7400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:37.164891 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126b847-57e6-486e-bbab-414ace6e7400 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:37.170786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefd796-550c-4af1-a163-406278b8453a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:37.171415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefd796-550c-4af1-a163-406278b8453a service placement] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:37.171699 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefd796-550c-4af1-a163-406278b8453a service placement] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:37.171976 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefd796-550c-4af1-a163-406278b8453a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:37.173776 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6aefd796-550c-4af1-a163-406278b8453a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:37.187830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefd796-550c-4af1-a163-406278b8453a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:37.195463 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefd796-550c-4af1-a163-406278b8453a service placement] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:37.204791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1403/2816] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:25:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:25:41.244827 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b939026-b757-4011-85b0-b49d9c0fd417 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:41.320516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b939026-b757-4011-85b0-b49d9c0fd417 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:41.326452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:41.327110 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:41.327396 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:41.327676 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] PATH_INFO: `/v3/users/01789f5a54314436abbc60f227011827` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:41.329608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01789f5a54314436abbc60f227011827)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:41.331780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:41.520063 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01789f5a54314436abbc60f227011827'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:41.520638 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01789f5a54314436abbc60f227011827'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:41.527461 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01789f5a54314436abbc60f227011827'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:41.607987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01789f5a54314436abbc60f227011827'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:41.608641 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01789f5a54314436abbc60f227011827'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:41.627150 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1414/2817] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:25:41 2022] DELETE /identity/v3/users/01789f5a54314436abbc60f227011827 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:25:41.631387 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2097-2581-4c01-b6ef-978ce7301dd1 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:41.635783 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2097-2581-4c01-b6ef-978ce7301dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:41.635783 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2097-2581-4c01-b6ef-978ce7301dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:41.650748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-42ae2097-2581-4c01-b6ef-978ce7301dd1 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:41.652629 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-42ae2097-2581-4c01-b6ef-978ce7301dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:41.711382 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1404/2818] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:41.721592 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:41.723021 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:41.787381 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27c9fa-764b-4399-9c1a-3ecc62178e71 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:41.792342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fd135-55b7-4ed2-a3ae-2f45f5dfbb9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:41.794839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fd135-55b7-4ed2-a3ae-2f45f5dfbb9f service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:41.794839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fd135-55b7-4ed2-a3ae-2f45f5dfbb9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:41.794839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fd135-55b7-4ed2-a3ae-2f45f5dfbb9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:41.804874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fd135-55b7-4ed2-a3ae-2f45f5dfbb9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:41.808052 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fd135-55b7-4ed2-a3ae-2f45f5dfbb9f service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:41.819085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1415/2819] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:25:42.223405 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae2097-2581-4c01-b6ef-978ce7301dd1 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:42.225046 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-42ae2097-2581-4c01-b6ef-978ce7301dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:42.252573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae2097-2581-4c01-b6ef-978ce7301dd1 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:42.259168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:42.259168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:42.259168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:42.259168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] PATH_INFO: `/v3/projects/229ae4f08a1e410496c798a50c043d38` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:42.260624 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] RBAC: Authorizing `identity:delete_project(project_id=229ae4f08a1e410496c798a50c043d38)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:42.263142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:42.379863 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '229ae4f08a1e410496c798a50c043d38', 'request_id': 'req-be591b82-7712-469c-851f-a9a879aa811b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:42.403292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '229ae4f08a1e410496c798a50c043d38', 'request_id': 'req-be591b82-7712-469c-851f-a9a879aa811b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:42.483509 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '229ae4f08a1e410496c798a50c043d38', 'request_id': 'req-be591b82-7712-469c-851f-a9a879aa811b'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:42.499041 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] The token cache is being invalidate because project 229ae4f08a1e410496c798a50c043d38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:25:42.499596 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:42.500394 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-be591b82-7712-469c-851f-a9a879aa811b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:25:42.502308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1405/2820] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:25:42 2022] DELETE /identity/v3/projects/229ae4f08a1e410496c798a50c043d38 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:25:50.017799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fd135-55b7-4ed2-a3ae-2f45f5dfbb9f service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:50.019313 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e11fd135-55b7-4ed2-a3ae-2f45f5dfbb9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:50.051280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:50.105500 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fd135-55b7-4ed2-a3ae-2f45f5dfbb9f service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:50.180489 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b3ef2-9e07-488d-85cb-036c77ee9fca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:50.181408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b3ef2-9e07-488d-85cb-036c77ee9fca service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:50.182420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b3ef2-9e07-488d-85cb-036c77ee9fca service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:50.182877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b3ef2-9e07-488d-85cb-036c77ee9fca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:50.185682 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d18b3ef2-9e07-488d-85cb-036c77ee9fca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:50.253782 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b3ef2-9e07-488d-85cb-036c77ee9fca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:50.255314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b3ef2-9e07-488d-85cb-036c77ee9fca service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:50.264859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1416/2821] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:25:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:25:52.494736 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:53.730847 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9472a-7045-4ad5-9dbd-5ed55d524ce5 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:53.731601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9472a-7045-4ad5-9dbd-5ed55d524ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:53.731601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9472a-7045-4ad5-9dbd-5ed55d524ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:53.751088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-60b9472a-7045-4ad5-9dbd-5ed55d524ce5 None None] MFA Rules not processed for user `e2b1f1c4bb2747e0845cdc2991e53b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:25:53.754528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-60b9472a-7045-4ad5-9dbd-5ed55d524ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:53.759385 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1406/2822] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:25:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:25:54.065839 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b3ef2-9e07-488d-85cb-036c77ee9fca service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:25:54.067225 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d18b3ef2-9e07-488d-85cb-036c77ee9fca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:54.124765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b3ef2-9e07-488d-85cb-036c77ee9fca service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:25:54.128490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd76615-6acf-4980-8b43-dd22c3602080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:25:54.129070 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd76615-6acf-4980-8b43-dd22c3602080 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:25:54.129353 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd76615-6acf-4980-8b43-dd22c3602080 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:25:54.129629 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd76615-6acf-4980-8b43-dd22c3602080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:25:54.131392 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3fd76615-6acf-4980-8b43-dd22c3602080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:25:54.141294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd76615-6acf-4980-8b43-dd22c3602080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:25:54.142711 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd76615-6acf-4980-8b43-dd22c3602080 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:25:54.146800 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:25:54.150713 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1417/2823] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:25:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:26:00.512471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9472a-7045-4ad5-9dbd-5ed55d524ce5 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:26:00.522992 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9472a-7045-4ad5-9dbd-5ed55d524ce5 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:26:00.527615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc462e59-773e-4d73-81e2-fd046bedce60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:26:00.527615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc462e59-773e-4d73-81e2-fd046bedce60 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:26:00.527615 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc462e59-773e-4d73-81e2-fd046bedce60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:26:00.527996 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc462e59-773e-4d73-81e2-fd046bedce60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:26:00.529514 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fc462e59-773e-4d73-81e2-fd046bedce60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:26:00.592131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc462e59-773e-4d73-81e2-fd046bedce60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:26:00.593637 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc462e59-773e-4d73-81e2-fd046bedce60 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:26:00.603260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1407/2824] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:26:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:26:10.598934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=33 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:26:59.714318 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd76615-6acf-4980-8b43-dd22c3602080 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:26:59.715781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3fd76615-6acf-4980-8b43-dd22c3602080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:26:59.726735 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd76615-6acf-4980-8b43-dd22c3602080 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:26:59.730515 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a43a8-1003-44b7-903b-bd455319375e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:26:59.731117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a43a8-1003-44b7-903b-bd455319375e service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:26:59.731394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a43a8-1003-44b7-903b-bd455319375e service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:26:59.731664 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a43a8-1003-44b7-903b-bd455319375e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:26:59.733411 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-202a43a8-1003-44b7-903b-bd455319375e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:26:59.797573 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a43a8-1003-44b7-903b-bd455319375e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:26:59.799013 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a43a8-1003-44b7-903b-bd455319375e service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:26:59.809442 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1418/2825] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:27:07.264312 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc462e59-773e-4d73-81e2-fd046bedce60 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:27:07.265873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fc462e59-773e-4d73-81e2-fd046bedce60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:27:07.282072 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc462e59-773e-4d73-81e2-fd046bedce60 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:27:07.287296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:27:07.287942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:27:07.288227 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:27:07.288503 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] PATH_INFO: `/v3/users/30f0002d2a5a4950b2f1aa771f33bbc1` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:27:07.303058 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] RBAC: Authorizing `identity:delete_user(user_id=30f0002d2a5a4950b2f1aa771f33bbc1)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:27:07.305008 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:27:07.490479 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30f0002d2a5a4950b2f1aa771f33bbc1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:07.497435 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30f0002d2a5a4950b2f1aa771f33bbc1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:07.582781 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30f0002d2a5a4950b2f1aa771f33bbc1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:07.583917 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30f0002d2a5a4950b2f1aa771f33bbc1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:07.596491 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30f0002d2a5a4950b2f1aa771f33bbc1'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:07.603183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1408/2826] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:27:07 2022] DELETE /identity/v3/users/30f0002d2a5a4950b2f1aa771f33bbc1 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:27:07.842420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a43a8-1003-44b7-903b-bd455319375e service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:27:07.853477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a43a8-1003-44b7-903b-bd455319375e service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:27:07.912408 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:27:07.913089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:27:07.913373 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:27:07.913653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] PATH_INFO: `/v3/projects/308e6cf086ae4cdea9c591ed2811c976` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:27:07.924599 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=308e6cf086ae4cdea9c591ed2811c976)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:27:07.926569 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:27:08.052477 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '308e6cf086ae4cdea9c591ed2811c976', 'request_id': 'req-73ded010-7a18-48b9-9aca-e5a72407c202'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.058077 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '308e6cf086ae4cdea9c591ed2811c976', 'request_id': 'req-73ded010-7a18-48b9-9aca-e5a72407c202'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.065177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '308e6cf086ae4cdea9c591ed2811c976', 'request_id': 'req-73ded010-7a18-48b9-9aca-e5a72407c202'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.142952 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] The token cache is being invalidate because project 308e6cf086ae4cdea9c591ed2811c976 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:27:08.143240 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.143869 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.146394 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1419/2827] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:27:07 2022] DELETE /identity/v3/projects/308e6cf086ae4cdea9c591ed2811c976 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:27:08.150049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:27:08.152154 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:27:08.166271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb742d-9881-408c-a4fd-9e914e8dc00c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:27:08.228652 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:27:08.229328 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:27:08.229617 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:27:08.229899 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] PATH_INFO: `/v3/users/a0719f10bb0c4a30b373ee6e2be69d93` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:27:08.243138 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0719f10bb0c4a30b373ee6e2be69d93)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:27:08.245108 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:27:08.375369 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0719f10bb0c4a30b373ee6e2be69d93'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.379790 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:27:08.383576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0719f10bb0c4a30b373ee6e2be69d93'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.434724 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0719f10bb0c4a30b373ee6e2be69d93'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.435346 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0719f10bb0c4a30b373ee6e2be69d93'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.449197 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0719f10bb0c4a30b373ee6e2be69d93'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.455089 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1409/2828] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:27:08 2022] DELETE /identity/v3/users/a0719f10bb0c4a30b373ee6e2be69d93 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:27:08.710903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:27:08.712344 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:27:08.731596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ded010-7a18-48b9-9aca-e5a72407c202 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:27:08.747238 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:27:08.792959 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:27:08.793623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:27:08.793924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:27:08.794259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] PATH_INFO: `/v3/projects/7694d36e7d80422ab3f1d49efc71ec10` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:27:08.805294 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7694d36e7d80422ab3f1d49efc71ec10)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:27:08.811217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:27:08.897906 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7694d36e7d80422ab3f1d49efc71ec10', 'request_id': 'req-68a7d3bf-155e-48bc-b939-4a50153f098c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.903425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7694d36e7d80422ab3f1d49efc71ec10', 'request_id': 'req-68a7d3bf-155e-48bc-b939-4a50153f098c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.909989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7694d36e7d80422ab3f1d49efc71ec10', 'request_id': 'req-68a7d3bf-155e-48bc-b939-4a50153f098c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.959598 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] The token cache is being invalidate because project 7694d36e7d80422ab3f1d49efc71ec10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:27:08.959821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.960431 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:27:08.961957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1420/2829] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:27:08 2022] DELETE /identity/v3/projects/7694d36e7d80422ab3f1d49efc71ec10 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:27:18.520346 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=10,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:27:18.956237 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:27:37.104044 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:27:37.106811 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:27:37.170232 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cd671-9643-4884-aa46-b4f25533f37b admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:27:37.221274 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536b7a4-1c2f-4458-969d-388128939acc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:27:37.221907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536b7a4-1c2f-4458-969d-388128939acc service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:27:37.222195 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536b7a4-1c2f-4458-969d-388128939acc service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:27:37.222477 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536b7a4-1c2f-4458-969d-388128939acc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:27:37.224279 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2536b7a4-1c2f-4458-969d-388128939acc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:27:37.295841 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536b7a4-1c2f-4458-969d-388128939acc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:27:37.297273 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536b7a4-1c2f-4458-969d-388128939acc service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:27:37.309263 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1410/2830] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:27:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:27:46.763568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:27:46.765207 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:27:46.777518 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7d3bf-155e-48bc-b939-4a50153f098c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:27:46.781066 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c60ff6-f49a-412c-bbec-1b6de5eb7de6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:27:46.781670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c60ff6-f49a-412c-bbec-1b6de5eb7de6 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:27:46.781948 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c60ff6-f49a-412c-bbec-1b6de5eb7de6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:27:46.782222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c60ff6-f49a-412c-bbec-1b6de5eb7de6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:27:46.783896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-10c60ff6-f49a-412c-bbec-1b6de5eb7de6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:27:46.797523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c60ff6-f49a-412c-bbec-1b6de5eb7de6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:27:46.798954 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c60ff6-f49a-412c-bbec-1b6de5eb7de6 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:27:46.811189 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1421/2831] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:27:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:27:47.304144 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=74 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:27:56.806125 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:27:57.413611 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85702b36-20b4-477a-8333-243f39737480 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:27:57.414318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85702b36-20b4-477a-8333-243f39737480 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:27:57.414318 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85702b36-20b4-477a-8333-243f39737480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:27:57.434049 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-85702b36-20b4-477a-8333-243f39737480 None None] MFA Rules not processed for user `e2b1f1c4bb2747e0845cdc2991e53b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:27:57.435271 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-85702b36-20b4-477a-8333-243f39737480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:27:57.489764 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1411/2832] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:27:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:28:07.483099 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=25 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:28:21.052301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c60ff6-f49a-412c-bbec-1b6de5eb7de6 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:28:21.052301 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-10c60ff6-f49a-412c-bbec-1b6de5eb7de6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:28:21.063308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c60ff6-f49a-412c-bbec-1b6de5eb7de6 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:28:21.070799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b42d4-f00a-45f1-b7a5-1cabcbc8cd51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:28:21.071434 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b42d4-f00a-45f1-b7a5-1cabcbc8cd51 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:28:21.071797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b42d4-f00a-45f1-b7a5-1cabcbc8cd51 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:28:21.072074 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b42d4-f00a-45f1-b7a5-1cabcbc8cd51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:28:21.073957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2c7b42d4-f00a-45f1-b7a5-1cabcbc8cd51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:28:21.091873 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b42d4-f00a-45f1-b7a5-1cabcbc8cd51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:28:21.094696 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b42d4-f00a-45f1-b7a5-1cabcbc8cd51 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:28:21.121119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1422/2833] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:28:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:28:21.234018 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85702b36-20b4-477a-8333-243f39737480 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:28:21.244305 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-85702b36-20b4-477a-8333-243f39737480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:28:21.313795 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85702b36-20b4-477a-8333-243f39737480 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:28:21.317949 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a997b0-ba82-4b85-a410-7e72c2fb9d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:28:21.318573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a997b0-ba82-4b85-a410-7e72c2fb9d96 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:28:21.318852 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a997b0-ba82-4b85-a410-7e72c2fb9d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:28:21.319124 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a997b0-ba82-4b85-a410-7e72c2fb9d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:28:21.321159 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f3a997b0-ba82-4b85-a410-7e72c2fb9d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:28:21.331777 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a997b0-ba82-4b85-a410-7e72c2fb9d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:28:21.334762 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a997b0-ba82-4b85-a410-7e72c2fb9d96 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:28:21.342429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1412/2834] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:28:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:28:31.102813 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:28:31.265300 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb5060-52cb-44c0-9357-4ad2646d3bb7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:28:31.265593 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb5060-52cb-44c0-9357-4ad2646d3bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:28:31.265871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb5060-52cb-44c0-9357-4ad2646d3bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:28:31.285413 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-90cb5060-52cb-44c0-9357-4ad2646d3bb7 None None] MFA Rules not processed for user `e3311c1ac5e349fa9ba83a5820171b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:28:31.286611 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-90cb5060-52cb-44c0-9357-4ad2646d3bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:28:31.338723 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=26 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:28:31.342533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1423/2835] 10.222.0.86 () {58 vars in 967 bytes} [Thu Jul 28 18:28:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:28:31.385183 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a997b0-ba82-4b85-a410-7e72c2fb9d96 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:28:31.386585 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f3a997b0-ba82-4b85-a410-7e72c2fb9d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:28:31.458207 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a997b0-ba82-4b85-a410-7e72c2fb9d96 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:28:31.464704 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58ccba-9cf7-4a17-af56-6566f53cc8bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3077d844257c48b3b4ac2db9177fb6df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:28:31.465354 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58ccba-9cf7-4a17-af56-6566f53cc8bb service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:28:31.465642 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58ccba-9cf7-4a17-af56-6566f53cc8bb service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:28:31.465925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58ccba-9cf7-4a17-af56-6566f53cc8bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:28:31.474482 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58ccba-9cf7-4a17-af56-6566f53cc8bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:28:31.476064 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58ccba-9cf7-4a17-af56-6566f53cc8bb service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:28:31.484231 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1413/2836] 10.222.0.86 () {60 vars in 1384 bytes} [Thu Jul 28 18:28:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 18:28:39.332890 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb5060-52cb-44c0-9357-4ad2646d3bb7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:28:39.334322 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-90cb5060-52cb-44c0-9357-4ad2646d3bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:28:39.410187 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb5060-52cb-44c0-9357-4ad2646d3bb7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:28:39.416244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be9b75-7e75-4d3e-89b3-d41332b06bb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:28:39.417052 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be9b75-7e75-4d3e-89b3-d41332b06bb4 service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:28:39.417440 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be9b75-7e75-4d3e-89b3-d41332b06bb4 service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:28:39.417821 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be9b75-7e75-4d3e-89b3-d41332b06bb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:28:39.419879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-84be9b75-7e75-4d3e-89b3-d41332b06bb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:28:39.433345 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be9b75-7e75-4d3e-89b3-d41332b06bb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:28:39.435039 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be9b75-7e75-4d3e-89b3-d41332b06bb4 service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:28:39.445490 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1424/2837] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:28:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:28:39.468429 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58ccba-9cf7-4a17-af56-6566f53cc8bb service swift] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:28:39.470872 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ad58ccba-9cf7-4a17-af56-6566f53cc8bb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:28:39.546297 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58ccba-9cf7-4a17-af56-6566f53cc8bb service swift] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:28:39.550966 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:28:39.551616 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:28:39.551902 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:28:39.552181 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] PATH_INFO: `/v3/users/799a2726f3694d39a70cbdaed5257751` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:28:39.553397 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=799a2726f3694d39a70cbdaed5257751)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:28:39.555972 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:28:39.725441 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '799a2726f3694d39a70cbdaed5257751'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:39.732871 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '799a2726f3694d39a70cbdaed5257751'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:39.780437 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '799a2726f3694d39a70cbdaed5257751'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:39.781088 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '799a2726f3694d39a70cbdaed5257751'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:39.793848 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '799a2726f3694d39a70cbdaed5257751'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:39.799799 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1414/2838] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:28:39 2022] DELETE /identity/v3/users/799a2726f3694d39a70cbdaed5257751 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:28:40.039884 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be9b75-7e75-4d3e-89b3-d41332b06bb4 service placement] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:28:40.050536 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be9b75-7e75-4d3e-89b3-d41332b06bb4 service placement] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:28:40.108947 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:28:40.109613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:28:40.109896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:28:40.110177 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] PATH_INFO: `/v3/projects/fe865ae79cfc41c1a2668f91e8cf0f70` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:28:40.111339 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe865ae79cfc41c1a2668f91e8cf0f70)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:28:40.113272 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:28:40.218857 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:28:40.227229 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe865ae79cfc41c1a2668f91e8cf0f70', 'request_id': 'req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:40.232691 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe865ae79cfc41c1a2668f91e8cf0f70', 'request_id': 'req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:40.238916 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe865ae79cfc41c1a2668f91e8cf0f70', 'request_id': 'req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:40.309130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] The token cache is being invalidate because project fe865ae79cfc41c1a2668f91e8cf0f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:28:40.309130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:40.309130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:40.309130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1425/2839] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:28:40 2022] DELETE /identity/v3/projects/fe865ae79cfc41c1a2668f91e8cf0f70 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:28:40.312221 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:28:40.313661 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:28:40.327945 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecde252-bb85-46c6-b82e-07ee4030af37 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:28:40.332908 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:28:40.385051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:28:40.385689 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:28:40.385971 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:28:40.386246 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] PATH_INFO: `/v3/users/55deab2889444c8ca686c922c10d44bc` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:28:40.387416 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55deab2889444c8ca686c922c10d44bc)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:28:40.389349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:28:40.577932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55deab2889444c8ca686c922c10d44bc'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:40.584938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55deab2889444c8ca686c922c10d44bc'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:40.654602 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55deab2889444c8ca686c922c10d44bc'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:40.655226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55deab2889444c8ca686c922c10d44bc'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:40.667474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55deab2889444c8ca686c922c10d44bc'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:40.673496 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1415/2840] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:28:40 2022] DELETE /identity/v3/users/55deab2889444c8ca686c922c10d44bc => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:28:40.934996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:28:40.936528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:28:40.950147 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1ca25-7296-46f3-9eef-34d3b01fcc53 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:28:41.009535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:28:41.010186 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:28:41.010521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:28:41.010799 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] PATH_INFO: `/v3/projects/b83a89a10e6c4099907b2998c7a7fefa` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:28:41.011986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b83a89a10e6c4099907b2998c7a7fefa)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:28:41.013924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:28:41.116511 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b83a89a10e6c4099907b2998c7a7fefa', 'request_id': 'req-b651390f-be69-46b9-b9ee-d37afcbcc143'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:41.122639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b83a89a10e6c4099907b2998c7a7fefa', 'request_id': 'req-b651390f-be69-46b9-b9ee-d37afcbcc143'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:41.128620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b83a89a10e6c4099907b2998c7a7fefa', 'request_id': 'req-b651390f-be69-46b9-b9ee-d37afcbcc143'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:41.190772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] The token cache is being invalidate because project b83a89a10e6c4099907b2998c7a7fefa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:28:41.190997 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:41.191623 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:28:41.193158 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1426/2841] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:28:40 2022] DELETE /identity/v3/projects/b83a89a10e6c4099907b2998c7a7fefa => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:28:50.669935 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:28:51.187644 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:29:43.092576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:29:43.097173 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:29:43.192068 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e66c1-1ef3-45ef-ad60-2355a001cf19 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:29:43.265497 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499f1ad-8c26-460d-91b4-82dd9049f746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:29:43.266780 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499f1ad-8c26-460d-91b4-82dd9049f746 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:43.266926 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499f1ad-8c26-460d-91b4-82dd9049f746 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:43.267212 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499f1ad-8c26-460d-91b4-82dd9049f746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:43.270011 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0499f1ad-8c26-460d-91b4-82dd9049f746 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:29:43.287454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499f1ad-8c26-460d-91b4-82dd9049f746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:29:43.287454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499f1ad-8c26-460d-91b4-82dd9049f746 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:29:43.298314 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1416/2842] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:29:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:29:49.673520 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:29:49.674885 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:29:49.733166 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651390f-be69-46b9-b9ee-d37afcbcc143 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:29:49.736804 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebac00d-800c-4ef7-b984-93b9b6284c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:29:49.737456 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebac00d-800c-4ef7-b984-93b9b6284c14 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:49.737768 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebac00d-800c-4ef7-b984-93b9b6284c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:49.738042 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebac00d-800c-4ef7-b984-93b9b6284c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:49.739894 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1ebac00d-800c-4ef7-b984-93b9b6284c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:29:49.801951 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebac00d-800c-4ef7-b984-93b9b6284c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:29:49.803379 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebac00d-800c-4ef7-b984-93b9b6284c14 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:29:49.813542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1427/2843] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:29:53.294370 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=62 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:29:53.391051 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499f1ad-8c26-460d-91b4-82dd9049f746 service nova] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:29:53.402409 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0499f1ad-8c26-460d-91b4-82dd9049f746 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:29:53.407421 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499f1ad-8c26-460d-91b4-82dd9049f746 service nova] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:29:53.412086 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1159a5-4799-4160-8293-1c7032eea202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:29:53.412682 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1159a5-4799-4160-8293-1c7032eea202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:53.412998 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1159a5-4799-4160-8293-1c7032eea202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:53.413349 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1159a5-4799-4160-8293-1c7032eea202 admin admin] PATH_INFO: `/v3/users/e2b1f1c4bb2747e0845cdc2991e53b09` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:53.414533 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1159a5-4799-4160-8293-1c7032eea202 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2b1f1c4bb2747e0845cdc2991e53b09)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:29:53.416462 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1159a5-4799-4160-8293-1c7032eea202 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:29:53.553930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-fa1159a5-4799-4160-8293-1c7032eea202 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2b1f1c4bb2747e0845cdc2991e53b09'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:29:53.561014 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-fa1159a5-4799-4160-8293-1c7032eea202 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2b1f1c4bb2747e0845cdc2991e53b09'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:29:53.617954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-fa1159a5-4799-4160-8293-1c7032eea202 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2b1f1c4bb2747e0845cdc2991e53b09'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:29:53.618576 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-fa1159a5-4799-4160-8293-1c7032eea202 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2b1f1c4bb2747e0845cdc2991e53b09'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:29:53.630922 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-fa1159a5-4799-4160-8293-1c7032eea202 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2b1f1c4bb2747e0845cdc2991e53b09'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:29:53.637153 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1417/2844] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:29:53 2022] DELETE /identity/v3/users/e2b1f1c4bb2747e0845cdc2991e53b09 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 18:29:53.859201 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebac00d-800c-4ef7-b984-93b9b6284c14 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:29:53.870881 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebac00d-800c-4ef7-b984-93b9b6284c14 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:29:53.927614 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:29:53.928262 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:53.928545 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:53.928824 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] PATH_INFO: `/v3/projects/104313ce72c94ea68c8cd3ee7f0cab2e` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:53.930029 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=104313ce72c94ea68c8cd3ee7f0cab2e)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:29:53.931995 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:29:54.052594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '104313ce72c94ea68c8cd3ee7f0cab2e', 'request_id': 'req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:29:54.054566 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:29:54.058308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '104313ce72c94ea68c8cd3ee7f0cab2e', 'request_id': 'req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:29:54.064437 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '104313ce72c94ea68c8cd3ee7f0cab2e', 'request_id': 'req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:29:54.154538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] The token cache is being invalidate because project 104313ce72c94ea68c8cd3ee7f0cab2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:29:54.154833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:29:54.155464 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e08661c3-9a3f-4cb6-bb66-9d08076f58eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:29:54.157016 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1428/2845] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:29:53 2022] DELETE /identity/v3/projects/104313ce72c94ea68c8cd3ee7f0cab2e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:29:54.162001 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f772fe-1ee5-4f94-b820-1924c355e0c3 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:54.162294 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f772fe-1ee5-4f94-b820-1924c355e0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:54.162573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f772fe-1ee5-4f94-b820-1924c355e0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:54.182090 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c4f772fe-1ee5-4f94-b820-1924c355e0c3 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:29:54.183307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c4f772fe-1ee5-4f94-b820-1924c355e0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:29:54.244060 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1418/2846] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:29:54.252586 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbdcab-fded-44b9-8e37-9381a95cf98a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:54.252876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbdcab-fded-44b9-8e37-9381a95cf98a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:54.253196 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbdcab-fded-44b9-8e37-9381a95cf98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:54.273556 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-37dbdcab-fded-44b9-8e37-9381a95cf98a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:29:54.274775 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-37dbdcab-fded-44b9-8e37-9381a95cf98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:29:54.282864 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1429/2847] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:29:54.287019 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f772fe-1ee5-4f94-b820-1924c355e0c3 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:29:54.296815 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f772fe-1ee5-4f94-b820-1924c355e0c3 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:29:54.301471 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1abb7-f494-45c6-8911-42db8720fa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:29:54.302082 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1abb7-f494-45c6-8911-42db8720fa50 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:54.302368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1abb7-f494-45c6-8911-42db8720fa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:54.302646 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1abb7-f494-45c6-8911-42db8720fa50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:54.303347 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1abb7-f494-45c6-8911-42db8720fa50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:29:54.303653 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1abb7-f494-45c6-8911-42db8720fa50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:29:54.305283 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1abb7-f494-45c6-8911-42db8720fa50 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:29:54.317761 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1419/2848] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:29:54.322000 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbdcab-fded-44b9-8e37-9381a95cf98a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:29:54.332221 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbdcab-fded-44b9-8e37-9381a95cf98a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:29:54.336956 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbab47-54f6-4ed0-aed2-dec6d3b21e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:29:54.337597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbab47-54f6-4ed0-aed2-dec6d3b21e72 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:54.337930 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbab47-54f6-4ed0-aed2-dec6d3b21e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:54.338192 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbab47-54f6-4ed0-aed2-dec6d3b21e72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:54.338899 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbab47-54f6-4ed0-aed2-dec6d3b21e72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:29:54.340895 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbab47-54f6-4ed0-aed2-dec6d3b21e72 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:29:54.402720 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1430/2849] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:29:54 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:29:54.406788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1abb7-f494-45c6-8911-42db8720fa50 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:29:54.417360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1abb7-f494-45c6-8911-42db8720fa50 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:29:54.474483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed589a-13c9-492f-a75d-62881bdd7da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:29:54.475029 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed589a-13c9-492f-a75d-62881bdd7da4 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:54.475317 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed589a-13c9-492f-a75d-62881bdd7da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:54.475595 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed589a-13c9-492f-a75d-62881bdd7da4 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:54.476260 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed589a-13c9-492f-a75d-62881bdd7da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:29:54.478198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed589a-13c9-492f-a75d-62881bdd7da4 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:29:54.486361 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:29:54.559990 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1420/2850] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:29:54 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:29:54.564905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbab47-54f6-4ed0-aed2-dec6d3b21e72 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:29:54.575738 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbab47-54f6-4ed0-aed2-dec6d3b21e72 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:29:54.581314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4bb50-e030-434e-9cdd-6bebf2632808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:29:54.581957 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4bb50-e030-434e-9cdd-6bebf2632808 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:54.582265 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4bb50-e030-434e-9cdd-6bebf2632808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:54.582547 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4bb50-e030-434e-9cdd-6bebf2632808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:54.583274 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4bb50-e030-434e-9cdd-6bebf2632808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:29:54.584949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4bb50-e030-434e-9cdd-6bebf2632808 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:29:54.599983 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1431/2851] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:29:54.604451 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed589a-13c9-492f-a75d-62881bdd7da4 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:29:54.616531 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed589a-13c9-492f-a75d-62881bdd7da4 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:29:54.621226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a3432-72a6-4d2f-a42e-fbaa83a75147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:29:54.621843 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a3432-72a6-4d2f-a42e-fbaa83a75147 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:54.622130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a3432-72a6-4d2f-a42e-fbaa83a75147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:54.622415 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a3432-72a6-4d2f-a42e-fbaa83a75147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:54.623122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a3432-72a6-4d2f-a42e-fbaa83a75147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:29:54.624714 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a3432-72a6-4d2f-a42e-fbaa83a75147 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:29:54.637407 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1421/2852] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:29:54.641447 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4bb50-e030-434e-9cdd-6bebf2632808 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:29:54.655130 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4bb50-e030-434e-9cdd-6bebf2632808 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:29:54.659718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1587e18-26c6-439c-b00c-5120545d5461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:29:54.660382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1587e18-26c6-439c-b00c-5120545d5461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:54.660662 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1587e18-26c6-439c-b00c-5120545d5461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:54.660942 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1587e18-26c6-439c-b00c-5120545d5461 admin admin] PATH_INFO: `/v3/projects/a2c491628e54487b9073e8054314fb93/users/c587ba7f98fd494983b3b4c4566da516/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:54.676893 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1587e18-26c6-439c-b00c-5120545d5461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2c491628e54487b9073e8054314fb93, user_id=c587ba7f98fd494983b3b4c4566da516, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:29:54.679570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1587e18-26c6-439c-b00c-5120545d5461 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:29:54.745710 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1432/2853] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:29:54 2022] PUT /identity/v3/projects/a2c491628e54487b9073e8054314fb93/users/c587ba7f98fd494983b3b4c4566da516/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:29:54.757198 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbc97b-73ee-4156-a99c-9450b5cdfca7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:54.757495 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbc97b-73ee-4156-a99c-9450b5cdfca7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:54.757779 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbc97b-73ee-4156-a99c-9450b5cdfca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:54.785412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9efbc97b-73ee-4156-a99c-9450b5cdfca7 None None] MFA Rules not processed for user `c587ba7f98fd494983b3b4c4566da516`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:29:54.786643 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9efbc97b-73ee-4156-a99c-9450b5cdfca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:29:54.852739 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1422/2854] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:29:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:29:54.864452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1587e18-26c6-439c-b00c-5120545d5461 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:29:54.866227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e1587e18-26c6-439c-b00c-5120545d5461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:29:54.938793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1587e18-26c6-439c-b00c-5120545d5461 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:29:54.942634 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c321c3d-42c0-4c11-a749-bde4301ceb93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:29:54.943263 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c321c3d-42c0-4c11-a749-bde4301ceb93 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:29:54.943549 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c321c3d-42c0-4c11-a749-bde4301ceb93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:29:54.943826 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c321c3d-42c0-4c11-a749-bde4301ceb93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:29:54.952426 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c321c3d-42c0-4c11-a749-bde4301ceb93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:29:54.954135 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c321c3d-42c0-4c11-a749-bde4301ceb93 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:29:54.963081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1433/2855] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:29:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:30:04.492639 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbc97b-73ee-4156-a99c-9450b5cdfca7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:04.554480 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbc97b-73ee-4156-a99c-9450b5cdfca7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:04.559111 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:04.559752 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:04.560036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:04.560313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] PATH_INFO: `/v3/users/c587ba7f98fd494983b3b4c4566da516` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:04.561499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c587ba7f98fd494983b3b4c4566da516)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:04.563410 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:04.780391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c587ba7f98fd494983b3b4c4566da516'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:04.787953 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c587ba7f98fd494983b3b4c4566da516'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:04.864367 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c587ba7f98fd494983b3b4c4566da516'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:04.865110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c587ba7f98fd494983b3b4c4566da516'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:04.870923 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:04.877748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c587ba7f98fd494983b3b4c4566da516'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:04.883880 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1423/2856] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:30:04 2022] DELETE /identity/v3/users/c587ba7f98fd494983b3b4c4566da516 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:04.888281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9de1-8d93-4b75-b12d-d6840c6eaaa4 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:04.888765 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9de1-8d93-4b75-b12d-d6840c6eaaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:04.888877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f9de1-8d93-4b75-b12d-d6840c6eaaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:04.908027 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5c9f9de1-8d93-4b75-b12d-d6840c6eaaa4 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:04.909327 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5c9f9de1-8d93-4b75-b12d-d6840c6eaaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:04.968117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1434/2857] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:04.978517 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:04.979932 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:05.036741 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f1532-2f20-41aa-8458-cdb7c25c4922 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:05.040315 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f70e2-ce7e-4c43-9ec3-68027629bf4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:05.040934 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f70e2-ce7e-4c43-9ec3-68027629bf4e service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:05.041240 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f70e2-ce7e-4c43-9ec3-68027629bf4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:05.041520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f70e2-ce7e-4c43-9ec3-68027629bf4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:05.050800 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f70e2-ce7e-4c43-9ec3-68027629bf4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:05.052230 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f70e2-ce7e-4c43-9ec3-68027629bf4e service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:05.061874 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1424/2858] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:30:05.487809 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f9de1-8d93-4b75-b12d-d6840c6eaaa4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:05.490028 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5c9f9de1-8d93-4b75-b12d-d6840c6eaaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:05.502594 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f9de1-8d93-4b75-b12d-d6840c6eaaa4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:05.507346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:05.508070 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:05.508346 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:05.508620 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] PATH_INFO: `/v3/projects/a2c491628e54487b9073e8054314fb93` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:05.510608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2c491628e54487b9073e8054314fb93)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:05.512528 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:05.529247 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:05.640589 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2c491628e54487b9073e8054314fb93', 'request_id': 'req-2ec082d3-b611-4bce-9409-11d9044dfd55'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:05.646420 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2c491628e54487b9073e8054314fb93', 'request_id': 'req-2ec082d3-b611-4bce-9409-11d9044dfd55'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:05.652567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2c491628e54487b9073e8054314fb93', 'request_id': 'req-2ec082d3-b611-4bce-9409-11d9044dfd55'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:05.719986 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] The token cache is being invalidate because project a2c491628e54487b9073e8054314fb93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:30:05.720281 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:05.720907 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2ec082d3-b611-4bce-9409-11d9044dfd55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:05.723157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1435/2859] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:30:05 2022] DELETE /identity/v3/projects/a2c491628e54487b9073e8054314fb93 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:05.728292 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b54d49-8474-431f-a3ab-27aca35fca7e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:05.728801 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b54d49-8474-431f-a3ab-27aca35fca7e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:05.728921 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b54d49-8474-431f-a3ab-27aca35fca7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:05.748249 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d5b54d49-8474-431f-a3ab-27aca35fca7e None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:05.749483 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d5b54d49-8474-431f-a3ab-27aca35fca7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:05.809473 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1425/2860] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:05.817327 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d79dd1-e8a8-4f1c-b362-504e463caef8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:05.817616 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d79dd1-e8a8-4f1c-b362-504e463caef8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:05.817896 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d79dd1-e8a8-4f1c-b362-504e463caef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:05.840299 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f0d79dd1-e8a8-4f1c-b362-504e463caef8 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:05.841559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f0d79dd1-e8a8-4f1c-b362-504e463caef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:05.850204 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1436/2861] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:05.854546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b54d49-8474-431f-a3ab-27aca35fca7e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:05.864900 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b54d49-8474-431f-a3ab-27aca35fca7e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:05.869515 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26a5e4-f2df-4220-9deb-53b447dd66b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:05.870121 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26a5e4-f2df-4220-9deb-53b447dd66b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:05.870402 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26a5e4-f2df-4220-9deb-53b447dd66b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:05.870673 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26a5e4-f2df-4220-9deb-53b447dd66b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:05.871364 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26a5e4-f2df-4220-9deb-53b447dd66b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:30:05.871664 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26a5e4-f2df-4220-9deb-53b447dd66b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:05.873261 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26a5e4-f2df-4220-9deb-53b447dd66b9 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:05.885010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1426/2862] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:30:05.889176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d79dd1-e8a8-4f1c-b362-504e463caef8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:05.898924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d79dd1-e8a8-4f1c-b362-504e463caef8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:05.903493 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719dbef-d82c-4a7b-8b3e-a4bbd5017e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:05.904090 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719dbef-d82c-4a7b-8b3e-a4bbd5017e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:05.904368 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719dbef-d82c-4a7b-8b3e-a4bbd5017e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:05.904640 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719dbef-d82c-4a7b-8b3e-a4bbd5017e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:05.905342 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719dbef-d82c-4a7b-8b3e-a4bbd5017e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:05.907220 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719dbef-d82c-4a7b-8b3e-a4bbd5017e93 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:06.019781 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1437/2863] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:30:05 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:30:06.023789 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26a5e4-f2df-4220-9deb-53b447dd66b9 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:06.034137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26a5e4-f2df-4220-9deb-53b447dd66b9 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:06.091057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ab0c8-9b04-4504-984a-844d2d8cbc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:06.091680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ab0c8-9b04-4504-984a-844d2d8cbc9f admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:06.091965 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ab0c8-9b04-4504-984a-844d2d8cbc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:06.092242 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ab0c8-9b04-4504-984a-844d2d8cbc9f admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:06.092938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ab0c8-9b04-4504-984a-844d2d8cbc9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:06.094938 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ab0c8-9b04-4504-984a-844d2d8cbc9f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:06.156219 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1427/2864] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:30:06 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:30:06.160429 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719dbef-d82c-4a7b-8b3e-a4bbd5017e93 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:06.171198 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719dbef-d82c-4a7b-8b3e-a4bbd5017e93 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:06.175950 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f883035-878f-43f7-9534-f7c357ef0701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:06.176605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f883035-878f-43f7-9534-f7c357ef0701 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:06.176886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f883035-878f-43f7-9534-f7c357ef0701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:06.177200 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f883035-878f-43f7-9534-f7c357ef0701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:06.177902 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f883035-878f-43f7-9534-f7c357ef0701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:06.179496 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f883035-878f-43f7-9534-f7c357ef0701 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:06.189562 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1438/2865] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:30:06.196036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ab0c8-9b04-4504-984a-844d2d8cbc9f admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:06.206451 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ab0c8-9b04-4504-984a-844d2d8cbc9f admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:06.211828 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9380a5-62e6-4bfa-8f15-0089a2d27f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:06.212444 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9380a5-62e6-4bfa-8f15-0089a2d27f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:06.212726 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9380a5-62e6-4bfa-8f15-0089a2d27f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:06.213031 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9380a5-62e6-4bfa-8f15-0089a2d27f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:06.213740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9380a5-62e6-4bfa-8f15-0089a2d27f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:06.215321 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9380a5-62e6-4bfa-8f15-0089a2d27f44 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:06.216684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:06.226587 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1428/2866] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 18:30:06.230793 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f883035-878f-43f7-9534-f7c357ef0701 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:06.240886 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f883035-878f-43f7-9534-f7c357ef0701 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:06.245560 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e726f-c92f-4fe1-8334-63b6b27b9d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:06.250772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e726f-c92f-4fe1-8334-63b6b27b9d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:06.250772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e726f-c92f-4fe1-8334-63b6b27b9d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:06.250772 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e726f-c92f-4fe1-8334-63b6b27b9d82 admin admin] PATH_INFO: `/v3/projects/a9db7b96f21f441bb275a26c2942a0e1/users/a44207d124074e34b0bb60fd4bc59259/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:06.261822 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e726f-c92f-4fe1-8334-63b6b27b9d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9db7b96f21f441bb275a26c2942a0e1, user_id=a44207d124074e34b0bb60fd4bc59259, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:06.264695 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e726f-c92f-4fe1-8334-63b6b27b9d82 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:06.349267 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1439/2867] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:30:06 2022] PUT /identity/v3/projects/a9db7b96f21f441bb275a26c2942a0e1/users/a44207d124074e34b0bb60fd4bc59259/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:06.359209 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a508bd-bf51-42f7-9b08-923f96873e8e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:06.359548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a508bd-bf51-42f7-9b08-923f96873e8e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:06.359832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a508bd-bf51-42f7-9b08-923f96873e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:06.388036 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-00a508bd-bf51-42f7-9b08-923f96873e8e None None] MFA Rules not processed for user `a44207d124074e34b0bb60fd4bc59259`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:06.389335 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-00a508bd-bf51-42f7-9b08-923f96873e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:06.454631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1429/2868] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:06.465684 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e726f-c92f-4fe1-8334-63b6b27b9d82 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:06.470567 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-163e726f-c92f-4fe1-8334-63b6b27b9d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:06.540829 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e726f-c92f-4fe1-8334-63b6b27b9d82 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:06.546619 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa784536-96d0-4fe1-8c54-68ce7002f1a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:06.547237 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa784536-96d0-4fe1-8c54-68ce7002f1a6 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:06.547523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa784536-96d0-4fe1-8c54-68ce7002f1a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:06.547879 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa784536-96d0-4fe1-8c54-68ce7002f1a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:06.556362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa784536-96d0-4fe1-8c54-68ce7002f1a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:06.558009 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa784536-96d0-4fe1-8c54-68ce7002f1a6 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:06.567597 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1440/2869] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:30:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:30:06.766412 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a508bd-bf51-42f7-9b08-923f96873e8e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:06.829883 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a508bd-bf51-42f7-9b08-923f96873e8e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:06.835475 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:06.836110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:06.836396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:06.836675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] PATH_INFO: `/v3/users/a44207d124074e34b0bb60fd4bc59259` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:06.837886 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a44207d124074e34b0bb60fd4bc59259)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:06.839796 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:06.962867 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a44207d124074e34b0bb60fd4bc59259'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:06.969604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a44207d124074e34b0bb60fd4bc59259'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:07.017728 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a44207d124074e34b0bb60fd4bc59259'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:07.018343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a44207d124074e34b0bb60fd4bc59259'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:07.031451 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a44207d124074e34b0bb60fd4bc59259'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:07.037884 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1430/2870] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:30:06 2022] DELETE /identity/v3/users/a44207d124074e34b0bb60fd4bc59259 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:07.042457 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28982138-557e-4121-9649-85e70b590250 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:07.047516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28982138-557e-4121-9649-85e70b590250 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:07.047516 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28982138-557e-4121-9649-85e70b590250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:07.061974 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-28982138-557e-4121-9649-85e70b590250 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:07.063173 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-28982138-557e-4121-9649-85e70b590250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:07.082596 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:07.122482 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1441/2871] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:07.132942 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:07.134675 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:07.141134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:07.193822 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7eb672-df92-425c-9ce7-b29ceb8c8bb7 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:07.197518 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cccf5-1501-45e1-996e-6cab2c551f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:07.198134 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cccf5-1501-45e1-996e-6cab2c551f95 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:07.198413 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cccf5-1501-45e1-996e-6cab2c551f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:07.198688 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cccf5-1501-45e1-996e-6cab2c551f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:07.207964 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cccf5-1501-45e1-996e-6cab2c551f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:07.209528 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cccf5-1501-45e1-996e-6cab2c551f95 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:07.219244 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1431/2872] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:30:07.562330 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28982138-557e-4121-9649-85e70b590250 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:07.563998 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-28982138-557e-4121-9649-85e70b590250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:07.576626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28982138-557e-4121-9649-85e70b590250 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:07.581543 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:07.582193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:07.582478 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:07.582758 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] PATH_INFO: `/v3/projects/a9db7b96f21f441bb275a26c2942a0e1` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:07.584022 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9db7b96f21f441bb275a26c2942a0e1)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:07.586268 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:07.696985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9db7b96f21f441bb275a26c2942a0e1', 'request_id': 'req-1302a576-79ab-45e7-a3d4-95f3d0f6a716'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:07.702615 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9db7b96f21f441bb275a26c2942a0e1', 'request_id': 'req-1302a576-79ab-45e7-a3d4-95f3d0f6a716'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:07.709202 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9db7b96f21f441bb275a26c2942a0e1', 'request_id': 'req-1302a576-79ab-45e7-a3d4-95f3d0f6a716'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:07.794270 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] The token cache is being invalidate because project a9db7b96f21f441bb275a26c2942a0e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:30:07.794487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:07.795140 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1302a576-79ab-45e7-a3d4-95f3d0f6a716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:07.796761 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1442/2873] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:30:07 2022] DELETE /identity/v3/projects/a9db7b96f21f441bb275a26c2942a0e1 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:07.802302 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d044f-76ba-4ae0-b4a6-9eeffebd09cd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:07.804125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d044f-76ba-4ae0-b4a6-9eeffebd09cd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:07.804125 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d044f-76ba-4ae0-b4a6-9eeffebd09cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:07.822545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-175d044f-76ba-4ae0-b4a6-9eeffebd09cd None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:07.823769 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-175d044f-76ba-4ae0-b4a6-9eeffebd09cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:07.884139 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1432/2874] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:07.893559 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a759e3a-9bb6-46eb-8fcc-681a3f09883a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:07.893848 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a759e3a-9bb6-46eb-8fcc-681a3f09883a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:07.894637 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a759e3a-9bb6-46eb-8fcc-681a3f09883a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:07.915308 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0a759e3a-9bb6-46eb-8fcc-681a3f09883a None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:07.916513 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0a759e3a-9bb6-46eb-8fcc-681a3f09883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:07.924314 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1443/2875] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:07.928454 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d044f-76ba-4ae0-b4a6-9eeffebd09cd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:07.938167 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d044f-76ba-4ae0-b4a6-9eeffebd09cd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:07.942758 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de93451-4041-43c4-bb00-64ec2f463981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:07.943366 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de93451-4041-43c4-bb00-64ec2f463981 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:07.943655 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de93451-4041-43c4-bb00-64ec2f463981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:07.943930 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de93451-4041-43c4-bb00-64ec2f463981 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:07.944627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de93451-4041-43c4-bb00-64ec2f463981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:30:07.944933 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de93451-4041-43c4-bb00-64ec2f463981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:07.946554 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de93451-4041-43c4-bb00-64ec2f463981 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:07.958589 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1433/2876] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:30:07.962550 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a759e3a-9bb6-46eb-8fcc-681a3f09883a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:07.971690 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a759e3a-9bb6-46eb-8fcc-681a3f09883a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:07.976989 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8624c35-a526-4cd6-96f9-1ecee46d3e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:07.977588 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8624c35-a526-4cd6-96f9-1ecee46d3e5f admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:07.977867 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8624c35-a526-4cd6-96f9-1ecee46d3e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:07.978139 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8624c35-a526-4cd6-96f9-1ecee46d3e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:07.978810 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8624c35-a526-4cd6-96f9-1ecee46d3e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:07.980670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8624c35-a526-4cd6-96f9-1ecee46d3e5f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:08.077183 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1444/2877] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:30:07 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:30:08.081496 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de93451-4041-43c4-bb00-64ec2f463981 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:08.093200 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de93451-4041-43c4-bb00-64ec2f463981 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:08.151084 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af95273-0f4f-4a13-8261-cf8cded03a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:08.151711 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af95273-0f4f-4a13-8261-cf8cded03a64 admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:08.151995 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af95273-0f4f-4a13-8261-cf8cded03a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:08.152275 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af95273-0f4f-4a13-8261-cf8cded03a64 admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:08.152954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af95273-0f4f-4a13-8261-cf8cded03a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:08.154954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af95273-0f4f-4a13-8261-cf8cded03a64 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:08.275959 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1434/2878] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:30:08 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:30:08.280137 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8624c35-a526-4cd6-96f9-1ecee46d3e5f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:08.290452 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8624c35-a526-4cd6-96f9-1ecee46d3e5f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:08.295003 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf439c-eb4a-46e0-b241-4e28261626a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:08.295608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf439c-eb4a-46e0-b241-4e28261626a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:08.295889 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf439c-eb4a-46e0-b241-4e28261626a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:08.296164 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf439c-eb4a-46e0-b241-4e28261626a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:08.296918 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf439c-eb4a-46e0-b241-4e28261626a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:08.298526 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf439c-eb4a-46e0-b241-4e28261626a6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:08.311083 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1445/2879] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:30:08.316855 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af95273-0f4f-4a13-8261-cf8cded03a64 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:08.330601 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af95273-0f4f-4a13-8261-cf8cded03a64 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:08.335925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c69c2-e5c3-43e5-93f8-b84b9dc08c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:08.336545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c69c2-e5c3-43e5-93f8-b84b9dc08c09 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:08.336832 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c69c2-e5c3-43e5-93f8-b84b9dc08c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:08.337142 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c69c2-e5c3-43e5-93f8-b84b9dc08c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:08.337963 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c69c2-e5c3-43e5-93f8-b84b9dc08c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:08.340550 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c69c2-e5c3-43e5-93f8-b84b9dc08c09 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:08.354391 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1435/2880] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:30:08.358671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf439c-eb4a-46e0-b241-4e28261626a6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:08.368985 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf439c-eb4a-46e0-b241-4e28261626a6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:08.373718 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310c55e-e815-484b-b872-6d14e79c1d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:08.374378 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310c55e-e815-484b-b872-6d14e79c1d3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:08.374661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310c55e-e815-484b-b872-6d14e79c1d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:08.374938 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310c55e-e815-484b-b872-6d14e79c1d3a admin admin] PATH_INFO: `/v3/projects/27717b5db6024de5bf9254bd17b31a18/users/430e11fc758246d7871f7a759f6f1519/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:08.390048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310c55e-e815-484b-b872-6d14e79c1d3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27717b5db6024de5bf9254bd17b31a18, user_id=430e11fc758246d7871f7a759f6f1519, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:08.392694 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310c55e-e815-484b-b872-6d14e79c1d3a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:08.471958 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1446/2881] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:30:08 2022] PUT /identity/v3/projects/27717b5db6024de5bf9254bd17b31a18/users/430e11fc758246d7871f7a759f6f1519/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:08.482128 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f44a4-cd13-4d57-a00e-a1ab668e6e28 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:08.482413 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f44a4-cd13-4d57-a00e-a1ab668e6e28 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:08.482684 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f44a4-cd13-4d57-a00e-a1ab668e6e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:08.510373 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:08.513304 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-526f44a4-cd13-4d57-a00e-a1ab668e6e28 None None] MFA Rules not processed for user `430e11fc758246d7871f7a759f6f1519`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:08.514512 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-526f44a4-cd13-4d57-a00e-a1ab668e6e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:08.577669 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1436/2882] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:08.587719 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310c55e-e815-484b-b872-6d14e79c1d3a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:08.631287 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:08.648722 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310c55e-e815-484b-b872-6d14e79c1d3a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:08.653633 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0c195-dadc-4b88-983a-57a6c35155c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:08.654258 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0c195-dadc-4b88-983a-57a6c35155c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:08.654542 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0c195-dadc-4b88-983a-57a6c35155c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:08.654848 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0c195-dadc-4b88-983a-57a6c35155c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:08.655608 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0c195-dadc-4b88-983a-57a6c35155c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:08.657553 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0c195-dadc-4b88-983a-57a6c35155c2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:08.767508 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1447/2883] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:30:08 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:30:08.772185 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f44a4-cd13-4d57-a00e-a1ab668e6e28 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:08.783101 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f44a4-cd13-4d57-a00e-a1ab668e6e28 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:08.843785 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dbd76-50d3-499f-abca-e85c13120d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:08.844545 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dbd76-50d3-499f-abca-e85c13120d3e admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:08.844690 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dbd76-50d3-499f-abca-e85c13120d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:08.844838 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dbd76-50d3-499f-abca-e85c13120d3e admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:08.845564 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6dbd76-50d3-499f-abca-e85c13120d3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:08.847476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6dbd76-50d3-499f-abca-e85c13120d3e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:08.939296 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1437/2884] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:30:08 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:30:08.944487 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0c195-dadc-4b88-983a-57a6c35155c2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:08.955461 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0c195-dadc-4b88-983a-57a6c35155c2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:08.960244 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92503a22-5ccd-451f-b8eb-82de997da3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:08.960859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92503a22-5ccd-451f-b8eb-82de997da3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:08.961176 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92503a22-5ccd-451f-b8eb-82de997da3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:08.961455 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92503a22-5ccd-451f-b8eb-82de997da3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:08.962157 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92503a22-5ccd-451f-b8eb-82de997da3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:08.963767 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92503a22-5ccd-451f-b8eb-82de997da3cf admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:08.973945 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1448/2885] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:30:08.978238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dbd76-50d3-499f-abca-e85c13120d3e admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:08.988993 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dbd76-50d3-499f-abca-e85c13120d3e admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:08.993680 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b29cd8-bdde-43c7-93a7-90db59814b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:08.994340 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b29cd8-bdde-43c7-93a7-90db59814b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:08.994627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b29cd8-bdde-43c7-93a7-90db59814b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:08.994907 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b29cd8-bdde-43c7-93a7-90db59814b91 admin admin] PATH_INFO: `/v3/projects/410d9354e8744943a31f575163bc58a7/users/f4da8b05f79441f08366f4eeea9d04e1/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:09.010342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b29cd8-bdde-43c7-93a7-90db59814b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=410d9354e8744943a31f575163bc58a7, user_id=f4da8b05f79441f08366f4eeea9d04e1, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:09.013057 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b29cd8-bdde-43c7-93a7-90db59814b91 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:09.081363 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1438/2886] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:30:08 2022] PUT /identity/v3/projects/410d9354e8744943a31f575163bc58a7/users/f4da8b05f79441f08366f4eeea9d04e1/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:09.085582 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92503a22-5ccd-451f-b8eb-82de997da3cf admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:09.095866 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92503a22-5ccd-451f-b8eb-82de997da3cf admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:09.152915 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985acaf-9b2c-41ee-8570-7ac3e6688300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:09.153570 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985acaf-9b2c-41ee-8570-7ac3e6688300 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:09.153855 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985acaf-9b2c-41ee-8570-7ac3e6688300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:09.154133 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985acaf-9b2c-41ee-8570-7ac3e6688300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:09.154834 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985acaf-9b2c-41ee-8570-7ac3e6688300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:09.156425 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985acaf-9b2c-41ee-8570-7ac3e6688300 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:09.167625 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1449/2887] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:30:09.171756 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b29cd8-bdde-43c7-93a7-90db59814b91 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:09.182887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b29cd8-bdde-43c7-93a7-90db59814b91 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:09.188130 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfea019-bdc7-4ba2-9ae5-ff76e73a8026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:09.188793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfea019-bdc7-4ba2-9ae5-ff76e73a8026 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:09.189120 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfea019-bdc7-4ba2-9ae5-ff76e73a8026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:09.189403 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfea019-bdc7-4ba2-9ae5-ff76e73a8026 admin admin] PATH_INFO: `/v3/projects/410d9354e8744943a31f575163bc58a7/users/f4da8b05f79441f08366f4eeea9d04e1/roles/61306055d76a4d4695ad0489edb99ba0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:09.191446 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfea019-bdc7-4ba2-9ae5-ff76e73a8026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=410d9354e8744943a31f575163bc58a7, user_id=f4da8b05f79441f08366f4eeea9d04e1, role_id=61306055d76a4d4695ad0489edb99ba0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:09.194143 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfea019-bdc7-4ba2-9ae5-ff76e73a8026 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:09.204748 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1439/2888] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:30:09 2022] PUT /identity/v3/projects/410d9354e8744943a31f575163bc58a7/users/f4da8b05f79441f08366f4eeea9d04e1/roles/61306055d76a4d4695ad0489edb99ba0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:09.215541 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb04436-36ef-4b81-af7f-dd4e44ec6d88 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:09.215837 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb04436-36ef-4b81-af7f-dd4e44ec6d88 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:09.216117 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb04436-36ef-4b81-af7f-dd4e44ec6d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:09.244987 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5bb04436-36ef-4b81-af7f-dd4e44ec6d88 None None] MFA Rules not processed for user `f4da8b05f79441f08366f4eeea9d04e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:09.246224 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5bb04436-36ef-4b81-af7f-dd4e44ec6d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:09.315521 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1450/2889] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:09.326567 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfea019-bdc7-4ba2-9ae5-ff76e73a8026 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:09.328548 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2bfea019-bdc7-4ba2-9ae5-ff76e73a8026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:09.387075 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfea019-bdc7-4ba2-9ae5-ff76e73a8026 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:09.390727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073550a-0726-42ca-b8f5-4f7b862adf95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:09.391342 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073550a-0726-42ca-b8f5-4f7b862adf95 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:09.391623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073550a-0726-42ca-b8f5-4f7b862adf95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:09.391898 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073550a-0726-42ca-b8f5-4f7b862adf95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:09.401450 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073550a-0726-42ca-b8f5-4f7b862adf95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:09.402881 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073550a-0726-42ca-b8f5-4f7b862adf95 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:09.406788 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:09.413321 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1440/2890] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:30:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:30:09.441820 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb04436-36ef-4b81-af7f-dd4e44ec6d88 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:09.484875 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:09.503944 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb04436-36ef-4b81-af7f-dd4e44ec6d88 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:09.508613 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:09.509284 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:09.509568 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:09.509844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] PATH_INFO: `/v3/users/430e11fc758246d7871f7a759f6f1519` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:09.511004 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=430e11fc758246d7871f7a759f6f1519)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:09.512910 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:09.726134 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '430e11fc758246d7871f7a759f6f1519'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:09.726714 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '430e11fc758246d7871f7a759f6f1519'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:09.733878 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '430e11fc758246d7871f7a759f6f1519'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:09.794504 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '430e11fc758246d7871f7a759f6f1519'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:09.795129 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '430e11fc758246d7871f7a759f6f1519'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:09.813538 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1451/2891] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:30:09 2022] DELETE /identity/v3/users/430e11fc758246d7871f7a759f6f1519 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:09.818194 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659bf74-1601-456e-8eab-f4ddd44acb76 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:09.818489 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659bf74-1601-456e-8eab-f4ddd44acb76 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:09.818771 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659bf74-1601-456e-8eab-f4ddd44acb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:09.837623 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9659bf74-1601-456e-8eab-f4ddd44acb76 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:09.838830 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9659bf74-1601-456e-8eab-f4ddd44acb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:09.898217 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1441/2892] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:09.908081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:09.909564 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:09.966700 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926fe60-d7a0-46a5-ac7b-68c7f167de45 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:09.970385 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37ee9a-7ab0-471a-b5bd-af9bb7f1fd45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:09.970996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37ee9a-7ab0-471a-b5bd-af9bb7f1fd45 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:09.971279 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37ee9a-7ab0-471a-b5bd-af9bb7f1fd45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:09.971557 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37ee9a-7ab0-471a-b5bd-af9bb7f1fd45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:09.981222 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37ee9a-7ab0-471a-b5bd-af9bb7f1fd45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:09.982646 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37ee9a-7ab0-471a-b5bd-af9bb7f1fd45 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:09.993362 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1452/2893] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:30:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:30:10.408327 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659bf74-1601-456e-8eab-f4ddd44acb76 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:10.410368 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9659bf74-1601-456e-8eab-f4ddd44acb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:10.433087 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659bf74-1601-456e-8eab-f4ddd44acb76 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:10.437873 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:10.438918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:10.438918 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:10.439150 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] PATH_INFO: `/v3/projects/27717b5db6024de5bf9254bd17b31a18` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:10.440353 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=27717b5db6024de5bf9254bd17b31a18)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:10.442214 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:10.537985 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27717b5db6024de5bf9254bd17b31a18', 'request_id': 'req-56dab410-cb6c-45cd-aabd-60206dfd0b3c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:10.543360 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27717b5db6024de5bf9254bd17b31a18', 'request_id': 'req-56dab410-cb6c-45cd-aabd-60206dfd0b3c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:10.588681 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27717b5db6024de5bf9254bd17b31a18', 'request_id': 'req-56dab410-cb6c-45cd-aabd-60206dfd0b3c'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:10.599786 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] The token cache is being invalidate because project 27717b5db6024de5bf9254bd17b31a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:30:10.600080 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:10.601765 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:10.602436 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1442/2894] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:30:10 2022] DELETE /identity/v3/projects/27717b5db6024de5bf9254bd17b31a18 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:10.606507 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37ee9a-7ab0-471a-b5bd-af9bb7f1fd45 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:10.616533 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37ee9a-7ab0-471a-b5bd-af9bb7f1fd45 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:10.670940 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4429b2-dbc0-492c-8285-23512a309e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:10.671574 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4429b2-dbc0-492c-8285-23512a309e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:10.671858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4429b2-dbc0-492c-8285-23512a309e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:10.672138 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4429b2-dbc0-492c-8285-23512a309e8f admin admin] PATH_INFO: `/v3/users/f4da8b05f79441f08366f4eeea9d04e1` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:10.673331 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4429b2-dbc0-492c-8285-23512a309e8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4da8b05f79441f08366f4eeea9d04e1)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:10.675309 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4429b2-dbc0-492c-8285-23512a309e8f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:10.677671 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:10.798049 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8a4429b2-dbc0-492c-8285-23512a309e8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4da8b05f79441f08366f4eeea9d04e1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:10.798662 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8a4429b2-dbc0-492c-8285-23512a309e8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4da8b05f79441f08366f4eeea9d04e1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:10.806382 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8a4429b2-dbc0-492c-8285-23512a309e8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4da8b05f79441f08366f4eeea9d04e1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:10.875031 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8a4429b2-dbc0-492c-8285-23512a309e8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4da8b05f79441f08366f4eeea9d04e1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:10.875661 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8a4429b2-dbc0-492c-8285-23512a309e8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4da8b05f79441f08366f4eeea9d04e1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:10.894833 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1453/2895] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:30:10 2022] DELETE /identity/v3/users/f4da8b05f79441f08366f4eeea9d04e1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:11.254177 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:11.255654 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:11.268813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dab410-cb6c-45cd-aabd-60206dfd0b3c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:11.326313 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:11.326887 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:11.327172 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:11.327449 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] PATH_INFO: `/v3/projects/410d9354e8744943a31f575163bc58a7` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:11.328863 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] RBAC: Authorizing `identity:delete_project(project_id=410d9354e8744943a31f575163bc58a7)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:11.330793 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:11.355129 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:11.459760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '410d9354e8744943a31f575163bc58a7', 'request_id': 'req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:11.465277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '410d9354e8744943a31f575163bc58a7', 'request_id': 'req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:11.537597 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '410d9354e8744943a31f575163bc58a7', 'request_id': 'req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:11.548991 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] The token cache is being invalidate because project 410d9354e8744943a31f575163bc58a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:30:11.549277 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:11.549899 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-f5e87485-8ed0-4e9d-ae9b-5a274126b96e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:30:11.552140 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1443/2896] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:30:11 2022] DELETE /identity/v3/projects/410d9354e8744943a31f575163bc58a7 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:11.559169 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922f1a4-5288-4986-b69b-7fbf572fa2ad None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:11.559486 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922f1a4-5288-4986-b69b-7fbf572fa2ad None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:11.559768 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922f1a4-5288-4986-b69b-7fbf572fa2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:11.583990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c922f1a4-5288-4986-b69b-7fbf572fa2ad None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:11.583990 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c922f1a4-5288-4986-b69b-7fbf572fa2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:11.638746 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1454/2897] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:11.648251 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21eff5-987e-4498-8bec-7ff1d52c88e2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:11.648544 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21eff5-987e-4498-8bec-7ff1d52c88e2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:11.648823 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21eff5-987e-4498-8bec-7ff1d52c88e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:11.669216 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-2b21eff5-987e-4498-8bec-7ff1d52c88e2 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:11.670422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2b21eff5-987e-4498-8bec-7ff1d52c88e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:11.678474 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1444/2898] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:11.682605 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922f1a4-5288-4986-b69b-7fbf572fa2ad None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:11.692280 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922f1a4-5288-4986-b69b-7fbf572fa2ad None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:11.696868 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce1c69-db41-4edd-9b02-9f2ca21fdf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:11.697529 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce1c69-db41-4edd-9b02-9f2ca21fdf66 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:11.697812 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce1c69-db41-4edd-9b02-9f2ca21fdf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:11.698084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce1c69-db41-4edd-9b02-9f2ca21fdf66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:11.698780 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce1c69-db41-4edd-9b02-9f2ca21fdf66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 18:30:11.699085 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce1c69-db41-4edd-9b02-9f2ca21fdf66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:11.700670 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce1c69-db41-4edd-9b02-9f2ca21fdf66 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:11.712659 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1455/2899] 10.222.0.86 () {58 vars in 1153 bytes} [Thu Jul 28 18:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 18:30:11.716760 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21eff5-987e-4498-8bec-7ff1d52c88e2 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:11.726455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21eff5-987e-4498-8bec-7ff1d52c88e2 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:11.731071 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d3509-e4c9-4c81-970a-addcc446d88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:11.731677 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d3509-e4c9-4c81-970a-addcc446d88c admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:11.731954 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d3509-e4c9-4c81-970a-addcc446d88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:11.732226 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d3509-e4c9-4c81-970a-addcc446d88c admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:11.732894 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d3509-e4c9-4c81-970a-addcc446d88c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:11.734813 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d3509-e4c9-4c81-970a-addcc446d88c admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:11.832023 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1445/2900] 10.222.0.86 () {60 vars in 1153 bytes} [Thu Jul 28 18:30:11 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 18:30:11.836227 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce1c69-db41-4edd-9b02-9f2ca21fdf66 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:11.846454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce1c69-db41-4edd-9b02-9f2ca21fdf66 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:11.903149 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391ee91-d4f4-4dbf-9591-ef6350426675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:11.903803 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ee91-d4f4-4dbf-9591-ef6350426675 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:11.904086 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ee91-d4f4-4dbf-9591-ef6350426675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:11.904365 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ee91-d4f4-4dbf-9591-ef6350426675 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:11.905092 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391ee91-d4f4-4dbf-9591-ef6350426675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:11.907081 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391ee91-d4f4-4dbf-9591-ef6350426675 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:11.995048 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:12.000669 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1456/2901] 10.222.0.86 () {60 vars in 1144 bytes} [Thu Jul 28 18:30:11 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 18:30:12.005499 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d3509-e4c9-4c81-970a-addcc446d88c admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:12.016111 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d3509-e4c9-4c81-970a-addcc446d88c admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:12.020925 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781a11c-af4a-4b6b-8380-6a48a0d8cdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:12.021573 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781a11c-af4a-4b6b-8380-6a48a0d8cdac admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:12.021860 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781a11c-af4a-4b6b-8380-6a48a0d8cdac admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:12.022137 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781a11c-af4a-4b6b-8380-6a48a0d8cdac admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:12.022844 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781a11c-af4a-4b6b-8380-6a48a0d8cdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:12.024467 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781a11c-af4a-4b6b-8380-6a48a0d8cdac admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:12.034520 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1446/2902] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:30:12.040937 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391ee91-d4f4-4dbf-9591-ef6350426675 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:12.051653 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391ee91-d4f4-4dbf-9591-ef6350426675 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:12.056259 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feebec49-cb77-4669-a12e-a2c831bb5dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:12.056871 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebec49-cb77-4669-a12e-a2c831bb5dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:12.057421 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebec49-cb77-4669-a12e-a2c831bb5dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:12.057874 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebec49-cb77-4669-a12e-a2c831bb5dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:12.058931 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feebec49-cb77-4669-a12e-a2c831bb5dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:12.060561 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feebec49-cb77-4669-a12e-a2c831bb5dcc admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:12.070880 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1457/2903] 10.222.0.86 () {58 vars in 1122 bytes} [Thu Jul 28 18:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 18:30:12.077396 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781a11c-af4a-4b6b-8380-6a48a0d8cdac admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:12.087928 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781a11c-af4a-4b6b-8380-6a48a0d8cdac admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:12.092737 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22609751-5314-45b8-b6f4-33e6a391f4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:12.093433 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22609751-5314-45b8-b6f4-33e6a391f4b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:12.093718 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22609751-5314-45b8-b6f4-33e6a391f4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:12.094016 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22609751-5314-45b8-b6f4-33e6a391f4b3 admin admin] PATH_INFO: `/v3/projects/f1e8613cb3f4463a8450d22995801d40/users/4649c6a25797428faacc2fd3857568df/roles/2dc5251f14e945e38ad1777a5f235180` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:12.109319 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22609751-5314-45b8-b6f4-33e6a391f4b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e8613cb3f4463a8450d22995801d40, user_id=4649c6a25797428faacc2fd3857568df, role_id=2dc5251f14e945e38ad1777a5f235180)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:12.112053 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22609751-5314-45b8-b6f4-33e6a391f4b3 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:12.171977 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1447/2904] 10.222.0.86 () {60 vars in 1483 bytes} [Thu Jul 28 18:30:12 2022] PUT /identity/v3/projects/f1e8613cb3f4463a8450d22995801d40/users/4649c6a25797428faacc2fd3857568df/roles/2dc5251f14e945e38ad1777a5f235180 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:30:12.177119 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c1bb5-b6f9-4510-9980-a451fb5134b5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:12.177409 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c1bb5-b6f9-4510-9980-a451fb5134b5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:12.177688 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c1bb5-b6f9-4510-9980-a451fb5134b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:12.196715 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-495c1bb5-b6f9-4510-9980-a451fb5134b5 None None] MFA Rules not processed for user `1dff148688b84b08aaa5c8689269444f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:12.197965 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-495c1bb5-b6f9-4510-9980-a451fb5134b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:12.256914 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1458/2905] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:12.266598 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22609751-5314-45b8-b6f4-33e6a391f4b3 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:12.268306 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-22609751-5314-45b8-b6f4-33e6a391f4b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:12.325229 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22609751-5314-45b8-b6f4-33e6a391f4b3 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:12.329122 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b18fc7-f63a-43ba-8115-58bace7a9806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:12.329732 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18fc7-f63a-43ba-8115-58bace7a9806 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:12.330110 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18fc7-f63a-43ba-8115-58bace7a9806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:12.330388 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18fc7-f63a-43ba-8115-58bace7a9806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:12.339546 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b18fc7-f63a-43ba-8115-58bace7a9806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:12.340905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b18fc7-f63a-43ba-8115-58bace7a9806 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:12.350961 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1448/2906] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:30:13.481388 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ce7a-b0b3-4c36-9795-581f82b16236 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:13.481905 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ce7a-b0b3-4c36-9795-581f82b16236 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:13.482014 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ce7a-b0b3-4c36-9795-581f82b16236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:13.510667 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e841ce7a-b0b3-4c36-9795-581f82b16236 None None] MFA Rules not processed for user `4649c6a25797428faacc2fd3857568df`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:13.511897 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e841ce7a-b0b3-4c36-9795-581f82b16236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:13.578657 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1459/2907] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:13.590032 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b18fc7-f63a-43ba-8115-58bace7a9806 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:13.591400 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-59b18fc7-f63a-43ba-8115-58bace7a9806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:13.648117 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b18fc7-f63a-43ba-8115-58bace7a9806 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:13.651727 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0c755-b20a-491a-8e84-bcda02a16ac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dbaf1e343474322b2567e2dfda99d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:13.652343 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0c755-b20a-491a-8e84-bcda02a16ac4 service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:13.652627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0c755-b20a-491a-8e84-bcda02a16ac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:13.652905 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0c755-b20a-491a-8e84-bcda02a16ac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:13.658176 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:13.662307 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0c755-b20a-491a-8e84-bcda02a16ac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:13.663740 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0c755-b20a-491a-8e84-bcda02a16ac4 service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:13.672558 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1449/2908] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:30:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:30:15.324882 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d747dca-e475-4e7e-9c07-a9f3a25616a8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:15.325384 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d747dca-e475-4e7e-9c07-a9f3a25616a8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:15.325523 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d747dca-e475-4e7e-9c07-a9f3a25616a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:15.345084 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7d747dca-e475-4e7e-9c07-a9f3a25616a8 None None] MFA Rules not processed for user `4649c6a25797428faacc2fd3857568df`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:30:15.346285 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7d747dca-e475-4e7e-9c07-a9f3a25616a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:15.353097 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1460/2909] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:30:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:30:15.958605 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0c755-b20a-491a-8e84-bcda02a16ac4 service cinder] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:15.960043 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b4f0c755-b20a-491a-8e84-bcda02a16ac4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:16.032285 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0c755-b20a-491a-8e84-bcda02a16ac4 service cinder] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:16.037911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbef2d5-7d2d-4030-a848-f4d82816fae6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:16.038525 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbef2d5-7d2d-4030-a848-f4d82816fae6 service placement] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:16.038807 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbef2d5-7d2d-4030-a848-f4d82816fae6 service placement] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:16.039083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbef2d5-7d2d-4030-a848-f4d82816fae6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:16.040826 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4bbef2d5-7d2d-4030-a848-f4d82816fae6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:16.052910 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbef2d5-7d2d-4030-a848-f4d82816fae6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:16.054455 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbef2d5-7d2d-4030-a848-f4d82816fae6 service placement] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:16.064476 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1450/2910] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:30:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:30:25.339797 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=68 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:26.060168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=35 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:29.319571 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d747dca-e475-4e7e-9c07-a9f3a25616a8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:29.321120 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7d747dca-e475-4e7e-9c07-a9f3a25616a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:29.398454 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d747dca-e475-4e7e-9c07-a9f3a25616a8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:29.404008 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23133bf-841c-4373-98f3-38053d045c2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d7bcb4e7c434622b650cdfc500a9680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:29.404639 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23133bf-841c-4373-98f3-38053d045c2f service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:29.404926 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23133bf-841c-4373-98f3-38053d045c2f service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:29.405242 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23133bf-841c-4373-98f3-38053d045c2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:29.407007 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c23133bf-841c-4373-98f3-38053d045c2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:29.422179 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23133bf-841c-4373-98f3-38053d045c2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:29.423996 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23133bf-841c-4373-98f3-38053d045c2f service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:29.436876 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1461/2911] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:30:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:30:39.431887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=34 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:30:41.172791 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbef2d5-7d2d-4030-a848-f4d82816fae6 service placement] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:30:41.237720 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbef2d5-7d2d-4030-a848-f4d82816fae6 service placement] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:30:41.243160 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba6040-c0a5-497b-bb76-195ee6677329 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:30:41.243792 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba6040-c0a5-497b-bb76-195ee6677329 service placement] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:30:41.244083 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba6040-c0a5-497b-bb76-195ee6677329 service placement] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:30:41.244365 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba6040-c0a5-497b-bb76-195ee6677329 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:30:41.247039 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-21ba6040-c0a5-497b-bb76-195ee6677329 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:30:41.281422 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba6040-c0a5-497b-bb76-195ee6677329 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:30:41.283030 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba6040-c0a5-497b-bb76-195ee6677329 service placement] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:30:41.295604 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1451/2912] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:30:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:30:51.290168 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=36 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:31:41.029660 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fb2ee-04cf-4d01-b0b5-df4a581a46ba None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:31:41.030193 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fb2ee-04cf-4d01-b0b5-df4a581a46ba None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:31:41.030321 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fb2ee-04cf-4d01-b0b5-df4a581a46ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:31:41.048844 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-bc7fb2ee-04cf-4d01-b0b5-df4a581a46ba None None] MFA Rules not processed for user `4649c6a25797428faacc2fd3857568df`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 18:31:41.050089 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bc7fb2ee-04cf-4d01-b0b5-df4a581a46ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:31:41.056949 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1462/2913] 10.222.0.86 () {58 vars in 958 bytes} [Thu Jul 28 18:31:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 18:31:41.298021 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba6040-c0a5-497b-bb76-195ee6677329 service placement] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:31:41.317350 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba6040-c0a5-497b-bb76-195ee6677329 service placement] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:31:41.321010 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892888f-ce37-4a0b-a856-673dfffb4963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b038336582a0435dbd47ad2c09cd23e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:31:41.321626 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892888f-ce37-4a0b-a856-673dfffb4963 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:31:41.321912 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892888f-ce37-4a0b-a856-673dfffb4963 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:31:41.322196 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892888f-ce37-4a0b-a856-673dfffb4963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:31:41.323962 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5892888f-ce37-4a0b-a856-673dfffb4963 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:31:41.346705 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892888f-ce37-4a0b-a856-673dfffb4963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:31:41.348131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892888f-ce37-4a0b-a856-673dfffb4963 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:31:41.356278 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1452/2914] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:31:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:31:51.043576 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:31:51.352627 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=15 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:32:59.407858 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fb2ee-04cf-4d01-b0b5-df4a581a46ba None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:32:59.440261 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fb2ee-04cf-4d01-b0b5-df4a581a46ba None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:32:59.444893 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:32:59.449877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:32:59.449877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:32:59.449877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] PATH_INFO: `/v3/users/4649c6a25797428faacc2fd3857568df` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:32:59.449877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4649c6a25797428faacc2fd3857568df)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:32:59.449877 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:32:59.625234 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4649c6a25797428faacc2fd3857568df'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:32:59.625887 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4649c6a25797428faacc2fd3857568df'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:32:59.632742 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4649c6a25797428faacc2fd3857568df'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:32:59.709217 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4649c6a25797428faacc2fd3857568df'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:32:59.709859 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4649c6a25797428faacc2fd3857568df'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:32:59.727391 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1463/2915] 10.222.0.86 () {58 vars in 1224 bytes} [Thu Jul 28 18:32:59 2022] DELETE /identity/v3/users/4649c6a25797428faacc2fd3857568df => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:32:59.979270 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892888f-ce37-4a0b-a856-673dfffb4963 service neutron] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:32:59.991344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892888f-ce37-4a0b-a856-673dfffb4963 service neutron] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:33:00.054696 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dff148688b84b08aaa5c8689269444f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2f95f7ebd0046e996236ac2938731ae', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:33:00.055344 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:33:00.055631 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:33:00.055911 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] PATH_INFO: `/v3/projects/f1e8613cb3f4463a8450d22995801d40` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:33:00.057131 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1e8613cb3f4463a8450d22995801d40)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:33:00.059104 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:33:00.166459 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1e8613cb3f4463a8450d22995801d40', 'request_id': 'req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:33:00.171862 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1e8613cb3f4463a8450d22995801d40', 'request_id': 'req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:33:00.257742 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1e8613cb3f4463a8450d22995801d40', 'request_id': 'req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:33:00.282308 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] The token cache is being invalidate because project f1e8613cb3f4463a8450d22995801d40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:33:00.282599 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:33:00.283238 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-7d1bc225-9bdf-4d93-b4b2-28ac2fb95cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:33:00.284784 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1453/2916] 10.222.0.86 () {58 vars in 1233 bytes} [Thu Jul 28 18:32:59 2022] DELETE /identity/v3/projects/f1e8613cb3f4463a8450d22995801d40 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:33:09.723903 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:33:10.280056 dv-cinder-846173-4-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:35:49.145470 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:35:49.146939 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:35:49.215243 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d050af-3d7e-45d2-835e-b3dec0a0b0b9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:35:49.273535 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab7966-0e5f-429d-b7f6-426e4a6de984 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b20564a83953433290aa3fabf884370c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bae648a61bc8421a947f6f4c30413b5d', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:35:49.274160 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab7966-0e5f-429d-b7f6-426e4a6de984 service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:35:49.274443 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab7966-0e5f-429d-b7f6-426e4a6de984 service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:35:49.274721 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab7966-0e5f-429d-b7f6-426e4a6de984 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:35:49.276486 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5cab7966-0e5f-429d-b7f6-426e4a6de984 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:35:49.287752 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab7966-0e5f-429d-b7f6-426e4a6de984 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:35:49.289609 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab7966-0e5f-429d-b7f6-426e4a6de984 service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:35:49.299924 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1464/2917] 10.222.0.86 () {60 vars in 1365 bytes} [Thu Jul 28 18:35:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:35:59.295626 dv-cinder-846173-4-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=58 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}